[pdm-devel] [PATCH datacenter-manager v3 00/19] add remote type based search and PBS node status panel to dashboard

[pdm-devel] [PATCH datacenter-manager v3 00/19] add remote type based search and PBS node status panel to dashboard

[Christian Ebner]

Extends the search capability to filter resource by remote type, so
this can be used to further filter the status when clicking the remote
type specific status panel in the dashboard as well.

Extend the PVE node status panel implementation to be reusabe for the
PBS node status and add a list of failed remotes to the status api
response, which allows to further discriminate errors by remote type
and show the type specific status message accordingly.

Changes since version 2:
- Includes also dashboard panel for datastore status

Changes since version 1 (thanks @Dominik for feedback):
- Reworked filtering helpers so the whole remote is passed as reference
  not just the remote type. This makes the filter easier to extend in
  the future.
- Completely reworked the datastore status panel, moving it to a dedicated
  component. Pass only the status information relevant for the respective
  remote type the node status component.

datacenter-manager:

Christian Ebner (19):
  server: fix small formatting issue via `cargo fmt`
  server: api: pass remote as reference to fetching helpers
  server: api: refactor filter logic for resource post gathering
  api: resources: new transient type for remote resource gathering
  server: api: add remote-type search category for resources
  pdm-api-types: extend resource status by list of failed remotes
  server: api: collect failed remotes list while getting status
  ui: dashboard: reimplement node status panel as dedicated component
  ui: dashboard: use new node status component
  ui: dashboard: extend node panel creation by remote type
  ui: dashboard: expose PBS nodes status panel
  pdm-api-types: introduce PBS datastore specific counters
  pdm-api-types/resources: extend datastore resources by config
    properties
  server: resources: extend the PBS resources by config properties
  server: resources: extend datastore status counters by multiple states
  pdm-api-types: extend status matching for PBS datastore resources
  pdm-api-types: extend resources by properties string generator method
  server: resources: add property matching for resources
  ui: dashboard: add panel for PBS datastore statistics

 lib/pdm-api-types/src/remotes.rs             |   3 +-
 lib/pdm-api-types/src/resource.rs            |  92 +++++++-
 server/src/api/resources.rs                  | 221 +++++++++++++++----
 server/src/metric_collection/top_entities.rs |   2 +-
 ui/src/dashboard/mod.rs                      | 191 ++++------------
 ui/src/dashboard/node_status_panel.rs        | 144 ++++++++++++
 ui/src/dashboard/pbs_datastores_panel.rs     | 175 +++++++++++++++
 ui/src/top_nav_bar.rs                        |   2 +-
 8 files changed, 640 insertions(+), 190 deletions(-)
 create mode 100644 ui/src/dashboard/node_status_panel.rs
 create mode 100644 ui/src/dashboard/pbs_datastores_panel.rs


Summary over all repositories:
  8 files changed, 640 insertions(+), 190 deletions(-)

-- 
Generated by git-murpp 0.8.1


_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 01/19] server: fix small formatting issue via `cargo fmt`

[Christian Ebner]

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/metric_collection/top_entities.rs | 2 +-
 ui/src/top_nav_bar.rs                        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/server/src/metric_collection/top_entities.rs b/server/src/metric_collection/top_entities.rs
index ea121ee..73a3e63 100644
--- a/server/src/metric_collection/top_entities.rs
+++ b/server/src/metric_collection/top_entities.rs
@@ -36,7 +36,7 @@ pub fn calculate_top(
     remotes: &HashMap<String, pdm_api_types::remotes::Remote>,
     timeframe: proxmox_rrd_api_types::RrdTimeframe,
     num: usize,
-    check_remote_privs: impl Fn(&str) -> bool
+    check_remote_privs: impl Fn(&str) -> bool,
 ) -> TopEntities {
     let mut guest_cpu = Vec::new();
     let mut node_cpu = Vec::new();
diff --git a/ui/src/top_nav_bar.rs b/ui/src/top_nav_bar.rs
index 74394a8..12dbb1e 100644
--- a/ui/src/top_nav_bar.rs
+++ b/ui/src/top_nav_bar.rs
@@ -19,8 +19,8 @@ use proxmox_yew_comp::{http_get, LanguageDialog, TaskViewer, ThemeDialog};
 
 use pwt_macros::builder;
 
-use pdm_api_types::RemoteUpid;
 use pbs_api_types::TaskListItem;
+use pdm_api_types::RemoteUpid;
 
 use crate::tasks::format_optional_remote_upid;
 use crate::widget::SearchBox;
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 02/19] server: api: pass remote as reference to fetching helpers

[Christian Ebner]

None of the helpers requires ownership of the remote, therefore pass
it as shared reference.

This is in preparation for passing the remote also to the search term
filtering to allow matching the remote type.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/api/resources.rs | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index d629c65..21143a8 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -255,7 +255,7 @@ pub(crate) async fn get_resources_impl(
         }
         let filter = filters.clone();
         let handle = tokio::spawn(async move {
-            let (mut resources, error) = match get_resources_for_remote(remote, max_age).await {
+            let (mut resources, error) = match get_resources_for_remote(&remote, max_age).await {
                 Ok(resources) => (resources, None),
                 Err(error) => {
                     tracing::debug!("failed to get resources from remote - {error:?}");
@@ -700,7 +700,7 @@ static CACHE: LazyLock<RwLock<HashMap<String, CachedResources>>> =
 ///
 /// If recent enough cached data is available, it is returned
 /// instead of calling out to the remote.
-async fn get_resources_for_remote(remote: Remote, max_age: u64) -> Result<Vec<Resource>, Error> {
+async fn get_resources_for_remote(remote: &Remote, max_age: u64) -> Result<Vec<Resource>, Error> {
     let remote_name = remote.id.to_owned();
     if let Some(cached_resource) = get_cached_resources(&remote_name, max_age) {
         Ok(cached_resource.resources)
@@ -756,7 +756,7 @@ fn update_cached_resources(remote: &str, resources: &[Resource], now: i64) {
 }
 
 /// Fetch remote resources and map to pdm-native data types.
-async fn fetch_remote_resource(remote: Remote) -> Result<Vec<Resource>, Error> {
+async fn fetch_remote_resource(remote: &Remote) -> Result<Vec<Resource>, Error> {
     let mut resources = Vec::new();
     let remote_name = remote.id.to_owned();
 
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 03/19] server: api: refactor filter logic for resource post gathering

[Christian Ebner]

Avoids to loop twice over the list of remote resources if a filter is
set, at the cost of checking the presence of the filter on each element.

In preparation to also pass along the remote to the remote search term
matching.

No functional changes intended.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/api/resources.rs | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 21143a8..a00742b 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -292,18 +292,21 @@ pub(crate) async fn get_resources_impl(
 
     let mut remote_resources = Vec::new();
     for handle in join_handles {
-        remote_resources.push(handle.await?);
-    }
-
-    if !filters.is_empty() {
-        remote_resources.retain(|res| {
-            if !res.resources.is_empty() {
-                return true;
-            }
-            filters.matches(|filter| {
-                remote_matches_search_term(&res.remote, Some(res.error.is_none()), filter)
-            })
-        })
+        let remote_resource = handle.await?;
+
+        if filters.is_empty() {
+            remote_resources.push(remote_resource);
+        } else if !remote_resource.resources.is_empty() {
+            remote_resources.push(remote_resource);
+        } else if filters.matches(|filter| {
+            remote_matches_search_term(
+                &remote_resource.remote,
+                Some(remote_resource.error.is_none()),
+                filter,
+            )
+        }) {
+            remote_resources.push(remote_resource);
+        }
     }
 
     Ok(remote_resources)
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 04/19] api: resources: new transient type for remote resource gathering

[Christian Ebner]

Introduce a new transient type to be returned by the get resources
implementation helper, which also includes the full remote object.
Since the remote object can store secret information, it must be
avoided to return it in the API.

Requires now the mapping of the transient type to the final
`RemoteResources` for the API response at the cost of additional
looping over the list of remote resources.

In preparation for adding search capabilities to filter remote
resources based on remote type.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/api/resources.rs | 44 +++++++++++++++++++++++++++----------
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index a00742b..793db5e 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -144,6 +144,24 @@ fn remote_matches_search_term(remote_name: &str, online: Option<bool>, term: &Se
     }
 }
 
+// Transient type for remote resources gathering and filtering on remote properties
+pub(crate) struct RemoteWithResources {
+    remote_name: String,
+    remote: Remote,
+    resources: Vec<Resource>,
+    error: Option<String>,
+}
+
+impl Into<RemoteResources> for RemoteWithResources {
+    fn into(self) -> RemoteResources {
+        RemoteResources {
+            remote: self.remote_name,
+            resources: self.resources,
+            error: self.error,
+        }
+    }
+}
+
 #[api(
     // FIXME:: see list-like API calls in resource routers, we probably want more fine-grained
     // checks..
@@ -183,7 +201,10 @@ pub async fn get_resources(
     search: Option<String>,
     rpcenv: &mut dyn RpcEnvironment,
 ) -> Result<Vec<RemoteResources>, Error> {
-    get_resources_impl(max_age, search, resource_type, Some(rpcenv)).await
+    let remotes_with_resources =
+        get_resources_impl(max_age, search, resource_type, Some(rpcenv)).await?;
+    let resources = remotes_with_resources.into_iter().map(Into::into).collect();
+    Ok(resources)
 }
 
 // helper to determine if the combination of search terms requires the results
@@ -219,7 +240,7 @@ pub(crate) async fn get_resources_impl(
     search: Option<String>,
     resource_type: Option<ResourceType>,
     rpcenv: Option<&mut dyn RpcEnvironment>,
-) -> Result<Vec<RemoteResources>, Error> {
+) -> Result<Vec<RemoteWithResources>, Error> {
     let user_info = CachedUserInfo::new()?;
     let mut opt_auth_id = None;
     if let Some(ref rpcenv) = rpcenv {
@@ -280,8 +301,9 @@ pub(crate) async fn get_resources_impl(
                 });
             }
 
-            RemoteResources {
-                remote: remote_name,
+            RemoteWithResources {
+                remote_name,
+                remote,
                 resources,
                 error,
             }
@@ -292,20 +314,20 @@ pub(crate) async fn get_resources_impl(
 
     let mut remote_resources = Vec::new();
     for handle in join_handles {
-        let remote_resource = handle.await?;
+        let remote_with_resources = handle.await?;
 
         if filters.is_empty() {
-            remote_resources.push(remote_resource);
-        } else if !remote_resource.resources.is_empty() {
-            remote_resources.push(remote_resource);
+            remote_resources.push(remote_with_resources);
+        } else if !remote_with_resources.resources.is_empty() {
+            remote_resources.push(remote_with_resources);
         } else if filters.matches(|filter| {
             remote_matches_search_term(
-                &remote_resource.remote,
-                Some(remote_resource.error.is_none()),
+                &remote_with_resources.remote_name,
+                Some(remote_with_resources.error.is_none()),
                 filter,
             )
         }) {
-            remote_resources.push(remote_resource);
+            remote_resources.push(remote_with_resources);
         }
     }
 
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 05/19] server: api: add remote-type search category for resources

[Christian Ebner]

Extend the current search capability for resources by adding the
`RemoteType` search category, allowing to selectively filter remotes
of type PVE and PBS.

Since the current remote filtering is only applied in case of remotes
only search, add an additional helper to pre-filter the to be fetched
resources by the remote type.

With this it is now possible to search, e.g. `remote-type:pbs` to
only get resources from PBS remotes, with the intention to provide
a search shortcut to show PBS remotes when clicking in the to be
added PBS status dashboard panel.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/api/resources.rs | 39 ++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 793db5e..b3f3ba2 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -56,6 +56,7 @@ enum MatchCategory {
     Status,
     Template,
     Remote,
+    RemoteType,
 }
 
 impl std::str::FromStr for MatchCategory {
@@ -69,6 +70,7 @@ impl std::str::FromStr for MatchCategory {
             "status" => MatchCategory::Status,
             "template" => MatchCategory::Template,
             "remote" => MatchCategory::Remote,
+            "remote-type" => MatchCategory::RemoteType,
             _ => bail!("invalid category"),
         };
         Ok(category)
@@ -88,11 +90,18 @@ impl MatchCategory {
                 (Ok(a), Ok(b)) => a == b,
                 _ => false,
             },
+            MatchCategory::RemoteType => match (
+                RemoteType::from_str(value),
+                RemoteType::from_str(search_term),
+            ) {
+                (Ok(a), Ok(b)) => a == b,
+                _ => false,
+            },
         }
     }
 }
 
-// returns None if we can't decide if it matches, currently only for the `Remote` category`
+// returns None if we can't decide if it matches, currently only for the `RemoteType` category
 fn resource_matches_search_term(
     remote_name: &str,
     resource: &Resource,
@@ -112,6 +121,7 @@ fn resource_matches_search_term(
                 _ => false,
             },
             MatchCategory::Remote => category.matches(remote_name, &term.value),
+            MatchCategory::RemoteType => return None,
         },
         Some(Err(_)) => false,
         None => {
@@ -122,7 +132,12 @@ fn resource_matches_search_term(
     Some(matches)
 }
 
-fn remote_matches_search_term(remote_name: &str, online: Option<bool>, term: &SearchTerm) -> bool {
+fn remote_matches_search_term(
+    remote_name: &str,
+    remote: &Remote,
+    online: Option<bool>,
+    term: &SearchTerm,
+) -> bool {
     match term.category.as_deref().map(|c| c.parse::<MatchCategory>()) {
         Some(Ok(category)) => match category {
             MatchCategory::Type => category.matches("remote", &term.value),
@@ -135,6 +150,7 @@ fn remote_matches_search_term(remote_name: &str, online: Option<bool>, term: &Se
                 None => true,
             },
             MatchCategory::Template => false,
+            MatchCategory::RemoteType => category.matches(&remote.ty.to_string(), &term.value),
         },
         Some(Err(_)) => false,
         None => {
@@ -144,6 +160,17 @@ fn remote_matches_search_term(remote_name: &str, online: Option<bool>, term: &Se
     }
 }
 
+fn remote_type_matches_search_term(remote_type: RemoteType, term: &SearchTerm) -> bool {
+    match term.category.as_deref().map(|c| c.parse::<MatchCategory>()) {
+        Some(Ok(category)) => match category {
+            MatchCategory::RemoteType => category.matches(&remote_type.to_string(), &term.value),
+            _ => true,
+        },
+        Some(Err(_)) => false,
+        None => true,
+    }
+}
+
 // Transient type for remote resources gathering and filtering on remote properties
 pub(crate) struct RemoteWithResources {
     remote_name: String,
@@ -269,8 +296,13 @@ pub(crate) async fn get_resources_impl(
             }
         }
 
+        if !filters.matches(|term| remote_type_matches_search_term(remote.ty, term)) {
+            continue;
+        }
+
         if remotes_only
-            && !filters.matches(|term| remote_matches_search_term(&remote_name, None, term))
+            && !filters
+                .matches(|term| remote_matches_search_term(&remote_name, &remote, None, term))
         {
             continue;
         }
@@ -323,6 +355,7 @@ pub(crate) async fn get_resources_impl(
         } else if filters.matches(|filter| {
             remote_matches_search_term(
                 &remote_with_resources.remote_name,
+                &remote_with_resources.remote,
                 Some(remote_with_resources.error.is_none()),
                 filter,
             )
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 06/19] pdm-api-types: extend resource status by list of failed remotes

[Christian Ebner]

Currently only the count of failed remotes is returned via the
status. Include the list of failed remotes including the name,
remote type and error message.

This will be used to extend the dashboard panel.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 lib/pdm-api-types/src/remotes.rs  |  3 ++-
 lib/pdm-api-types/src/resource.rs | 28 ++++++++++++++++++++++++++--
 2 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/lib/pdm-api-types/src/remotes.rs b/lib/pdm-api-types/src/remotes.rs
index 6318943..dd6afa6 100644
--- a/lib/pdm-api-types/src/remotes.rs
+++ b/lib/pdm-api-types/src/remotes.rs
@@ -39,10 +39,11 @@ pub struct NodeUrl {
 
 #[api]
 /// The type of a remote entry.
-#[derive(Clone, Copy, Debug, Eq, PartialEq, Deserialize, Serialize, Ord, PartialOrd)]
+#[derive(Clone, Copy, Default, Debug, Eq, PartialEq, Deserialize, Serialize, Ord, PartialOrd)]
 #[serde(rename_all = "lowercase")]
 pub enum RemoteType {
     /// A Proxmox VE node.
+    #[default]
     Pve,
     /// A Proxmox Backup Server node.
     Pbs,
diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index b219250..400f2af 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -5,7 +5,7 @@ use serde::{Deserialize, Serialize};
 
 use proxmox_schema::{api, ApiStringFormat, ApiType, EnumEntry, OneOfSchema, Schema, StringSchema};
 
-use super::remotes::REMOTE_ID_SCHEMA;
+use super::remotes::{RemoteType, REMOTE_ID_SCHEMA};
 
 #[api(
     "id-property": "id",
@@ -550,7 +550,16 @@ pub struct SdnZoneCount {
     pub unknown: u64,
 }
 
-#[api]
+#[api(
+    properties: {
+        "failed_remotes_list": {
+            type: Array,
+            items: {
+                type: FailedRemote,
+            },
+        }
+    }
+)]
 #[derive(Default, Serialize, Deserialize, Clone, PartialEq)]
 /// Describes the status of seen resources
 pub struct ResourcesStatus {
@@ -572,6 +581,21 @@ pub struct ResourcesStatus {
     pub pbs_nodes: NodeStatusCount,
     /// Status of PBS Datastores
     pub pbs_datastores: StorageStatusCount,
+    /// List of the failed remotes including type and error
+    #[serde(default, skip_serializing_if = "Vec::is_empty")]
+    pub failed_remotes_list: Vec<FailedRemote>,
+}
+
+#[api]
+#[derive(Default, Serialize, Deserialize, Clone, PartialEq)]
+/// Error information for a failed remote
+pub struct FailedRemote {
+    /// Name of the failed remote
+    pub name: String,
+    /// Error that occurred when querying remote resources
+    pub error: String,
+    /// Type of the failed remote
+    pub remote_type: RemoteType,
 }
 
 #[api(
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 07/19] server: api: collect failed remotes list while getting status

[Christian Ebner]

Include name, remote type and error message for failed remotes when
gathering status information, in order to be able to discriminate
errors by remote type for the dashboard.

Since the get_resources_impl has previously been adapted to return the
full remote object as well, utilize that over get_resources to have
the additional context.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 server/src/api/resources.rs | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index b3f3ba2..fb281f4 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -9,9 +9,9 @@ use futures::FutureExt;
 use pbs_api_types::{DataStoreStatusListItem, NodeStatus};
 use pdm_api_types::remotes::{Remote, RemoteType};
 use pdm_api_types::resource::{
-    PbsDatastoreResource, PbsNodeResource, PveLxcResource, PveNodeResource, PveQemuResource,
-    PveSdnResource, PveStorageResource, RemoteResources, Resource, ResourceType, ResourcesStatus,
-    SdnStatus, SdnZoneResource, TopEntities,
+    FailedRemote, PbsDatastoreResource, PbsNodeResource, PveLxcResource, PveNodeResource,
+    PveQemuResource, PveSdnResource, PveStorageResource, RemoteResources, Resource, ResourceType,
+    ResourcesStatus, SdnStatus, SdnZoneResource, TopEntities,
 };
 use pdm_api_types::subscription::{
     NodeSubscriptionInfo, RemoteSubscriptionState, RemoteSubscriptions, SubscriptionLevel,
@@ -395,15 +395,20 @@ pub async fn get_status(
     max_age: u64,
     rpcenv: &mut dyn RpcEnvironment,
 ) -> Result<ResourcesStatus, Error> {
-    let remotes = get_resources(max_age, None, None, rpcenv).await?;
+    let remotes_with_resources = get_resources_impl(max_age, None, None, Some(rpcenv)).await?;
     let mut counts = ResourcesStatus::default();
-    for remote in remotes {
-        if remote.error.is_some() {
+    for remote_with_resources in remotes_with_resources {
+        if let Some(err) = remote_with_resources.error {
             counts.failed_remotes += 1;
+            counts.failed_remotes_list.push(FailedRemote {
+                name: remote_with_resources.remote_name,
+                error: err.to_string(),
+                remote_type: remote_with_resources.remote.ty,
+            });
         } else {
             counts.remotes += 1;
         }
-        for resource in remote.resources {
+        for resource in remote_with_resources.resources {
             match resource {
                 Resource::PveStorage(r) => match r.status.as_str() {
                     "available" => counts.storages.available += 1,
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 08/19] ui: dashboard: reimplement node status panel as dedicated component

[Christian Ebner]

Reimplement the current node status panel as a dedicated component
for better encapsulation and input data handling, analogous to how
guest status or SDN status panels are handled.

Instead of passing in the whole status, only pass the state required
for the component to be rendered.

Already take into account that this will be used to render the PBS
node status as well.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 ui/src/dashboard/node_status_panel.rs | 144 ++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)
 create mode 100644 ui/src/dashboard/node_status_panel.rs

diff --git a/ui/src/dashboard/node_status_panel.rs b/ui/src/dashboard/node_status_panel.rs
new file mode 100644
index 0000000..e9e8c6b
--- /dev/null
+++ b/ui/src/dashboard/node_status_panel.rs
@@ -0,0 +1,144 @@
+use std::rc::Rc;
+
+use pdm_api_types::remotes::RemoteType;
+use pdm_api_types::resource::NodeStatusCount;
+use pdm_search::{Search, SearchTerm};
+use proxmox_yew_comp::Status;
+use pwt::{
+    css::{AlignItems, FlexFit, JustifyContent},
+    prelude::*,
+    widget::{Column, Fa},
+};
+use yew::{
+    virtual_dom::{VComp, VNode},
+    Properties,
+};
+
+use crate::search_provider::get_search_provider;
+
+use super::loading_column;
+
+#[derive(PartialEq, Clone, Properties)]
+pub struct NodeStatusPanel {
+    remote_type: RemoteType,
+    status: Option<NodeStatusCount>,
+    failed_remotes: usize,
+}
+
+impl NodeStatusPanel {
+    pub fn new(
+        remote_type: RemoteType,
+        status: Option<NodeStatusCount>,
+        failed_remotes: usize,
+    ) -> Self {
+        yew::props!(Self {
+            remote_type,
+            status,
+            failed_remotes,
+        })
+    }
+}
+
+impl From<NodeStatusPanel> for VNode {
+    fn from(value: NodeStatusPanel) -> Self {
+        let comp = VComp::new::<NodeStatusPanelComponent>(Rc::new(value), None);
+        VNode::from(comp)
+    }
+}
+
+pub struct NodeStatusPanelComponent {}
+
+impl yew::Component for NodeStatusPanelComponent {
+    type Message = Search;
+    type Properties = NodeStatusPanel;
+
+    fn create(_ctx: &yew::Context<Self>) -> Self {
+        Self {}
+    }
+
+    fn update(&mut self, ctx: &Context<Self>, msg: Self::Message) -> bool {
+        if let Some(provider) = get_search_provider(ctx) {
+            provider.search(msg);
+        }
+        false
+    }
+
+    fn view(&self, ctx: &yew::Context<Self>) -> yew::Html {
+        let props = ctx.props();
+
+        let (icon, status_msg, search_terms) = match &props.status {
+            Some(status) => map_status(status, props.remote_type, props.failed_remotes),
+            None => return loading_column().into(),
+        };
+
+        let column = Column::new()
+            .padding(4)
+            .class("pwt-pointer")
+            .class(FlexFit)
+            .class(AlignItems::Center)
+            .class(JustifyContent::Center)
+            .gap(2)
+            .onclick(ctx.link().callback({
+                let search_terms = search_terms.clone();
+                move |_| Search::with_terms(search_terms.clone())
+            }))
+            .onkeydown(ctx.link().batch_callback({
+                let search_terms = search_terms.clone();
+                move |event: KeyboardEvent| match event.key().as_str() {
+                    "Enter" | " " => Some(Search::with_terms(search_terms.clone())),
+                    _ => None,
+                }
+            }))
+            .with_child(icon.large_4x())
+            .with_child(status_msg);
+        column.into()
+    }
+}
+
+fn map_status(
+    status: &NodeStatusCount,
+    remote_type: RemoteType,
+    failed_remotes: usize,
+) -> (Fa, String, Vec<SearchTerm>) {
+    let mut search_terms = vec![
+        SearchTerm::new("node").category(Some("type")),
+        SearchTerm::new(remote_type.to_string()).category(Some("remote-type")),
+    ];
+    let (icon, status_msg) = match status {
+        NodeStatusCount {
+            online,
+            offline,
+            unknown,
+        } if *offline > 0 => {
+            search_terms.push(SearchTerm::new("offline").category(Some("status")));
+            (
+                Status::Error.into(),
+                tr!(
+                    "{0} of {1} nodes are offline",
+                    offline,
+                    online + offline + unknown,
+                ),
+            )
+        }
+        NodeStatusCount { unknown, .. } if *unknown > 0 => {
+            search_terms.push(SearchTerm::new("unknown").category(Some("status")));
+            (
+                Status::Warning.into(),
+                tr!("{0} nodes have an unknown status", unknown),
+            )
+        }
+        NodeStatusCount { online, .. } if failed_remotes > 0 => match remote_type {
+            RemoteType::Pve => (
+                Status::Unknown.into(),
+                tr!("{0} of an unknown number of nodes online", online),
+            ),
+            RemoteType::Pbs => (
+                Status::Error.into(),
+                tr!("{0} remotes failed", failed_remotes),
+            ),
+        },
+        NodeStatusCount { online, .. } => (Status::Success.into(), tr!("{0} nodes online", online)),
+    };
+
+    (icon, status_msg, search_terms)
+}
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 09/19] ui: dashboard: use new node status component

[Christian Ebner]

Given the reusable component implementation to render status of
PVE and PBS nodes, use it to create the PVE node status panel.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 ui/src/dashboard/mod.rs | 112 +++++++++-------------------------------
 1 file changed, 25 insertions(+), 87 deletions(-)

diff --git a/ui/src/dashboard/mod.rs b/ui/src/dashboard/mod.rs
index f54c509..8a678af 100644
--- a/ui/src/dashboard/mod.rs
+++ b/ui/src/dashboard/mod.rs
@@ -10,7 +10,7 @@ use yew::{
     Component,
 };
 
-use proxmox_yew_comp::{http_get, EditWindow, Status};
+use proxmox_yew_comp::{http_get, EditWindow};
 use pwt::{
     css::{AlignItems, FlexDirection, FlexFit, FlexWrap, JustifyContent},
     prelude::*,
@@ -25,16 +25,11 @@ use pwt::{
     AsyncPool,
 };
 
-use pdm_api_types::{
-    remotes::RemoteType,
-    resource::{NodeStatusCount, ResourcesStatus},
-    TaskStatistics,
-};
+use pdm_api_types::{remotes::RemoteType, resource::ResourcesStatus, TaskStatistics};
 use pdm_client::types::TopEntity;
-use pdm_search::{Search, SearchTerm};
 use proxmox_client::ApiResponseData;
 
-use crate::{pve::GuestType, remotes::AddWizard, search_provider::get_search_provider, RemoteList};
+use crate::{pve::GuestType, remotes::AddWizard, RemoteList};
 
 mod top_entities;
 pub use top_entities::TopEntities;
@@ -48,6 +43,9 @@ use remote_panel::RemotePanel;
 mod guest_panel;
 use guest_panel::GuestPanel;
 
+mod node_status_panel;
+use node_status_panel::NodeStatusPanel;
+
 mod sdn_zone_panel;
 use sdn_zone_panel::SdnZonePanel;
 
@@ -123,7 +121,6 @@ pub enum Msg {
     ForceReload,
     UpdateConfig(DashboardConfig),
     ConfigWindow(bool),
-    Search(Search),
 }
 
 struct StatisticsOptions {
@@ -158,79 +155,28 @@ impl PdmDashboard {
             .into()
     }
 
-    fn create_node_panel(&self, ctx: &yew::Context<Self>, icon: &str, title: String) -> Panel {
-        let mut search_terms = vec![SearchTerm::new("node").category(Some("type"))];
-        let (status_icon, text): (Fa, String) = match &self.status {
-            Some(status) => {
-                match status.pve_nodes {
-                    NodeStatusCount {
-                        online,
-                        offline,
-                        unknown,
-                    } if offline > 0 => {
-                        search_terms.push(SearchTerm::new("offline").category(Some("status")));
-                        (
-                            Status::Error.into(),
-                            tr!(
-                                "{0} of {1} nodes are offline",
-                                offline,
-                                online + offline + unknown,
-                            ),
-                        )
-                    }
-                    NodeStatusCount { unknown, .. } if unknown > 0 => {
-                        search_terms.push(SearchTerm::new("unknown").category(Some("status")));
-                        (
-                            Status::Warning.into(),
-                            tr!("{0} nodes have an unknown status", unknown),
-                        )
-                    }
-                    // FIXME, get more detailed status about the failed remotes (name, type, error)?
-                    NodeStatusCount { online, .. } if status.failed_remotes > 0 => (
-                        Status::Unknown.into(),
-                        tr!("{0} of an unknown number of nodes online", online),
-                    ),
-                    NodeStatusCount { online, .. } => {
-                        (Status::Success.into(), tr!("{0} nodes online", online))
-                    }
-                }
-            }
-            None => (Status::Unknown.into(), String::new()),
+    fn create_node_panel(&self, icon: &str, title: String) -> Panel {
+        let (nodes_status, failed_remotes) = match &self.status {
+            Some(status) => (
+                Some(status.pve_nodes.clone()),
+                status
+                    .failed_remotes_list
+                    .iter()
+                    .filter(|item| item.remote_type == RemoteType::Pve)
+                    .count(),
+            ),
+            None => (None, 0),
         };
-
-        let loading = self.status.is_none();
-        let search = Search::with_terms(search_terms);
         Panel::new()
             .flex(1.0)
             .width(300)
             .title(self.create_title_with_icon(icon, title))
             .border(true)
-            .with_child(
-                Column::new()
-                    .padding(4)
-                    .class("pwt-pointer")
-                    .onclick(ctx.link().callback({
-                        let search = search.clone();
-                        move |_| Msg::Search(search.clone())
-                    }))
-                    .onkeydown(ctx.link().batch_callback({
-                        let search = search.clone();
-                        move |event: KeyboardEvent| match event.key().as_str() {
-                            "Enter" | " " => Some(Msg::Search(search.clone())),
-                            _ => None,
-                        }
-                    }))
-                    .class(FlexFit)
-                    .class(AlignItems::Center)
-                    .class(JustifyContent::Center)
-                    .gap(2)
-                    .with_child(if loading {
-                        html! {<i class={"pwt-loading-icon"} />}
-                    } else {
-                        status_icon.large_4x().into()
-                    })
-                    .with_optional_child((!loading).then_some(text)),
-            )
+            .with_child(NodeStatusPanel::new(
+                RemoteType::Pve,
+                nodes_status,
+                failed_remotes,
+            ))
     }
 
     fn create_guest_panel(&self, guest_type: GuestType) -> Panel {
@@ -501,12 +447,6 @@ impl Component for PdmDashboard {
                 self.show_config_window = false;
                 true
             }
-            Msg::Search(search_term) => {
-                if let Some(provider) = get_search_provider(ctx) {
-                    provider.search(search_term.into());
-                }
-                false
-            }
         }
     }
 
@@ -563,11 +503,9 @@ impl Component for PdmDashboard {
                             )
                             .with_child(RemotePanel::new(self.status.clone())),
                     )
-                    .with_child(self.create_node_panel(
-                        ctx,
-                        "building",
-                        tr!("Virtual Environment Nodes"),
-                    ))
+                    .with_child(
+                        self.create_node_panel("building", tr!("Virtual Environment Nodes")),
+                    )
                     .with_child(self.create_guest_panel(GuestType::Qemu))
                     .with_child(self.create_guest_panel(GuestType::Lxc))
                     // FIXME: add PBS support
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 10/19] ui: dashboard: extend node panel creation by remote type

[Christian Ebner]

Make the node panel instantiation logic reusable for the PBS remotes
by passing the remote type as additional parameter, switching the
rendered node status information based on that.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 ui/src/dashboard/mod.rs | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/ui/src/dashboard/mod.rs b/ui/src/dashboard/mod.rs
index 8a678af..d90d56c 100644
--- a/ui/src/dashboard/mod.rs
+++ b/ui/src/dashboard/mod.rs
@@ -155,25 +155,30 @@ impl PdmDashboard {
             .into()
     }
 
-    fn create_node_panel(&self, icon: &str, title: String) -> Panel {
+    fn create_node_panel(&self, icon: &str, title: String, remote_type: RemoteType) -> Panel {
         let (nodes_status, failed_remotes) = match &self.status {
-            Some(status) => (
-                Some(status.pve_nodes.clone()),
-                status
+            Some(status) => {
+                let nodes_status = match remote_type {
+                    RemoteType::Pve => Some(status.pve_nodes.clone()),
+                    RemoteType::Pbs => Some(status.pbs_nodes.clone()),
+                };
+                let failed_remotes = status
                     .failed_remotes_list
                     .iter()
-                    .filter(|item| item.remote_type == RemoteType::Pve)
-                    .count(),
-            ),
+                    .filter(|item| item.remote_type == remote_type)
+                    .count();
+                (nodes_status, failed_remotes)
+            }
             None => (None, 0),
         };
+
         Panel::new()
             .flex(1.0)
             .width(300)
             .title(self.create_title_with_icon(icon, title))
             .border(true)
             .with_child(NodeStatusPanel::new(
-                RemoteType::Pve,
+                remote_type,
                 nodes_status,
                 failed_remotes,
             ))
@@ -503,9 +508,11 @@ impl Component for PdmDashboard {
                             )
                             .with_child(RemotePanel::new(self.status.clone())),
                     )
-                    .with_child(
-                        self.create_node_panel("building", tr!("Virtual Environment Nodes")),
-                    )
+                    .with_child(self.create_node_panel(
+                        "building",
+                        tr!("Virtual Environment Nodes"),
+                        RemoteType::Pve,
+                    ))
                     .with_child(self.create_guest_panel(GuestType::Qemu))
                     .with_child(self.create_guest_panel(GuestType::Lxc))
                     // FIXME: add PBS support
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 11/19] ui: dashboard: expose PBS nodes status panel

[Christian Ebner]

Analogous to the node status information shown for PVE nodes, allows
to get a fast overview if all PBS remotes/nodes are reachable or not.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- no changes

 ui/src/dashboard/mod.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ui/src/dashboard/mod.rs b/ui/src/dashboard/mod.rs
index d90d56c..d3da03d 100644
--- a/ui/src/dashboard/mod.rs
+++ b/ui/src/dashboard/mod.rs
@@ -515,12 +515,12 @@ impl Component for PdmDashboard {
                     ))
                     .with_child(self.create_guest_panel(GuestType::Qemu))
                     .with_child(self.create_guest_panel(GuestType::Lxc))
-                    // FIXME: add PBS support
-                    //.with_child(self.create_node_panel(
-                    //    "building-o",
-                    //    tr!("Backup Server Nodes"),
-                    //    &self.status.pbs_nodes,
-                    //))
+                    .with_child(self.create_node_panel(
+                        "building-o",
+                        tr!("Backup Server Nodes"),
+                        RemoteType::Pbs,
+                    ))
+                    // FIXME: add further PBS support
                     //.with_child(
                     //    Panel::new()
                     //        .flex(1.0)
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 12/19] pdm-api-types: introduce PBS datastore specific counters

[Christian Ebner]

Instead of using the storage counters, which are to limited to cover
counts of e.g. high usage stores, datastores which are in maintenance
mode ecc., introduce a dedicated type and add extended fields.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 lib/pdm-api-types/src/resource.rs | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index 400f2af..6c0125c 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -538,6 +538,28 @@ pub struct StorageStatusCount {
     pub unknown: u64,
 }
 
+#[api]
+#[derive(Default, Serialize, Deserialize, Clone, PartialEq)]
+/// Amount of Proxmox Backup Server datastores with certain state
+pub struct PbsDatastoreStatusCount {
+    /// Amount of online datastores
+    pub online: u64,
+    /// Amount of datastores which are in a maintenance mode
+    pub in_maintenance: Option<u64>,
+    /// Amount of datastores which have high datastore usage
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub high_usage: Option<u64>,
+    /// Amount of datastores in unknown state
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub unknown: Option<u64>,
+    /// Amount of removable datastores
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub removable: Option<u64>,
+    /// Amount of datastores with S3 backend
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub s3_backend: Option<u64>,
+}
+
 #[api]
 #[derive(Default, Serialize, Deserialize, Clone, PartialEq)]
 /// Amount of SDN zones in certain states
@@ -580,7 +602,7 @@ pub struct ResourcesStatus {
     /// Status of PBS Nodes
     pub pbs_nodes: NodeStatusCount,
     /// Status of PBS Datastores
-    pub pbs_datastores: StorageStatusCount,
+    pub pbs_datastores: PbsDatastoreStatusCount,
     /// List of the failed remotes including type and error
     #[serde(default, skip_serializing_if = "Vec::is_empty")]
     pub failed_remotes_list: Vec<FailedRemote>,
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 13/19] pdm-api-types/resources: extend datastore resources by config properties

[Christian Ebner]

Add currently configured datastore properties to the PBS datastore
resource definition. This will allow to identify datastores being
currently maintained, as well as their backing device and backend
type.

Derives the `Default` trait for easier object creation in case of
missing config information.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 lib/pdm-api-types/src/resource.rs | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index 6c0125c..b779825 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -460,7 +460,7 @@ pub struct PbsNodeResource {
 }
 
 #[api]
-#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
+#[derive(Clone, Debug, Default, Deserialize, Serialize, PartialEq)]
 #[serde(rename_all = "kebab-case")]
 /// PBS datastore resource
 pub struct PbsDatastoreResource {
@@ -472,6 +472,17 @@ pub struct PbsDatastoreResource {
     pub disk: u64,
     /// Datastore name
     pub name: String,
+    /// Datastore contents disk usage
+    pub usage: f64,
+    /// Datastore maintenance mode
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub maintenance: Option<String>,
+    /// Datastore backing device
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub backing_device: Option<String>,
+    /// Datastore backend type
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub backend_type: Option<String>,
 }
 
 #[api(
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 14/19] server: resources: extend the PBS resources by config properties

[Christian Ebner]

At the cost of one additional api call, fetch the datastore list
which includes datastore config properties. Use this to map the
additional information to the `PbsDatastoreResource`.

With the intend to show further datastore information on the
dashboard.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 server/src/api/resources.rs | 69 ++++++++++++++++++++++++++++++++-----
 1 file changed, 60 insertions(+), 9 deletions(-)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index fb281f4..793abfc 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -6,7 +6,9 @@ use anyhow::{bail, format_err, Error};
 use futures::future::join_all;
 use futures::FutureExt;
 
-use pbs_api_types::{DataStoreStatusListItem, NodeStatus};
+use pbs_api_types::{
+    DataStoreStatusListItem, DatastoreBackendConfig, DatastoreBackendType, NodeStatus,
+};
 use pdm_api_types::remotes::{Remote, RemoteType};
 use pdm_api_types::resource::{
     FailedRemote, PbsDatastoreResource, PbsNodeResource, PveLxcResource, PveNodeResource,
@@ -841,8 +843,18 @@ async fn fetch_remote_resource(remote: &Remote) -> Result<Vec<Resource>, Error>
             let status = client.node_status().await?;
             resources.push(map_pbs_node_status(&remote_name, status));
 
+            let datastores = client.list_datastores().await?;
+            let datastore_map: HashMap<String, pbs_api_types::DataStoreConfig> = datastores
+                .into_iter()
+                .map(|store| (store.name.clone(), store))
+                .collect();
+
             for datastore_usage in client.datastore_usage().await? {
-                resources.push(map_pbs_datastore_status(&remote_name, datastore_usage));
+                resources.push(map_pbs_datastore_status(
+                    &remote_name,
+                    datastore_usage,
+                    &datastore_map,
+                ));
             }
         }
     }
@@ -997,13 +1009,52 @@ fn map_pbs_node_status(remote: &str, status: NodeStatus) -> Resource {
     })
 }
 
-fn map_pbs_datastore_status(remote: &str, status: DataStoreStatusListItem) -> Resource {
-    Resource::PbsDatastore(PbsDatastoreResource {
-        id: format!("remote/{remote}/datastore/{}", status.store),
-        name: status.store,
-        maxdisk: status.total.unwrap_or_default(),
-        disk: status.used.unwrap_or_default(),
-    })
+fn map_pbs_datastore_status(
+    remote: &str,
+    status: DataStoreStatusListItem,
+    datastore_map: &HashMap<String, pbs_api_types::DataStoreConfig>,
+) -> Resource {
+    let maxdisk = status.total.unwrap_or_default();
+    let disk = status.used.unwrap_or_default();
+
+    let usage = if maxdisk > 0 {
+        disk as f64 / maxdisk as f64
+    } else {
+        0.0
+    };
+
+    if let Some(store_config) = datastore_map.get(&status.store) {
+        let mut backend_type = None;
+        if let Some(store_backend) = &store_config.backend {
+            match store_backend.parse::<DatastoreBackendConfig>() {
+                Ok(backend_config) => {
+                    if let Some(DatastoreBackendType::S3) = backend_config.ty {
+                        backend_type = Some(DatastoreBackendType::S3.to_string())
+                    }
+                }
+                Err(_) => backend_type = Some("unknown".to_string()),
+            }
+        }
+        Resource::PbsDatastore(PbsDatastoreResource {
+            id: format!("remote/{remote}/datastore/{}", status.store),
+            name: status.store,
+            maxdisk,
+            disk,
+            usage,
+            maintenance: store_config.maintenance_mode.clone(),
+            backing_device: store_config.backing_device.clone(),
+            backend_type,
+        })
+    } else {
+        Resource::PbsDatastore(PbsDatastoreResource {
+            id: format!("remote/{remote}/datastore/{}", status.store),
+            name: status.store,
+            maxdisk,
+            disk,
+            usage,
+            ..Default::default()
+        })
+    }
 }
 
 #[cfg(test)]
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 15/19] server: resources: extend datastore status counters by multiple states

[Christian Ebner]

Account for some more states which will be shown in the datastore
status panel of the dashboard.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 lib/pdm-api-types/src/resource.rs |  3 +++
 server/src/api/resources.rs       | 25 +++++++++++++++++++++++--
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index b779825..cc467d5 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -7,6 +7,9 @@ use proxmox_schema::{api, ApiStringFormat, ApiType, EnumEntry, OneOfSchema, Sche
 
 use super::remotes::{RemoteType, REMOTE_ID_SCHEMA};
 
+/// High PBS datastore usage threshold
+pub const PBS_DATASTORE_HIGH_USAGE_THRESHOLD: f64 = 0.75;
+
 #[api(
     "id-property": "id",
     "id-schema": {
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 793abfc..3b629d1 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -13,7 +13,7 @@ use pdm_api_types::remotes::{Remote, RemoteType};
 use pdm_api_types::resource::{
     FailedRemote, PbsDatastoreResource, PbsNodeResource, PveLxcResource, PveNodeResource,
     PveQemuResource, PveSdnResource, PveStorageResource, RemoteResources, Resource, ResourceType,
-    ResourcesStatus, SdnStatus, SdnZoneResource, TopEntities,
+    ResourcesStatus, SdnStatus, SdnZoneResource, TopEntities, PBS_DATASTORE_HIGH_USAGE_THRESHOLD,
 };
 use pdm_api_types::subscription::{
     NodeSubscriptionInfo, RemoteSubscriptionState, RemoteSubscriptions, SubscriptionLevel,
@@ -450,7 +450,28 @@ pub async fn get_status(
                 }
                 // FIXME better status for pbs/datastores
                 Resource::PbsNode(_) => counts.pbs_nodes.online += 1,
-                Resource::PbsDatastore(_) => counts.pbs_datastores.available += 1,
+                Resource::PbsDatastore(r) => {
+                    if r.maintenance.is_none() {
+                        counts.pbs_datastores.online += 1;
+                    } else {
+                        *counts.pbs_datastores.in_maintenance.get_or_insert_default() += 1;
+                    }
+                    if r.usage > PBS_DATASTORE_HIGH_USAGE_THRESHOLD {
+                        *counts.pbs_datastores.high_usage.get_or_insert_default() += 1;
+                    }
+                    if r.backing_device.is_some() {
+                        *counts.pbs_datastores.removable.get_or_insert_default() += 1;
+                    }
+                    match r.backend_type.as_deref() {
+                        Some("s3") => {
+                            *counts.pbs_datastores.s3_backend.get_or_insert_default() += 1;
+                        }
+                        Some("unknown") => {
+                            *counts.pbs_datastores.unknown.get_or_insert_default() += 1;
+                        }
+                        _ => (),
+                    }
+                }
             }
         }
     }
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 16/19] pdm-api-types: extend status matching for PBS datastore resources

[Christian Ebner]

Distinguish between datastores being online or being in a maintenance
mode. Used to filter resources based on status for the PBS datastore
dashboard panel.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 lib/pdm-api-types/src/resource.rs | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index cc467d5..7f74bf5 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -98,7 +98,13 @@ impl Resource {
                     "offline"
                 }
             }
-            Resource::PbsDatastore(_) => "online",
+            Resource::PbsDatastore(r) => {
+                if r.maintenance.is_none() {
+                    "online"
+                } else {
+                    "in-maintenance"
+                }
+            }
         }
     }
 }
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 17/19] pdm-api-types: extend resources by properties string generator method

[Christian Ebner]

Generates a list of comma separated values of properties to be
matched by the search functionality.

This will allow to define several properties on a resource which one
can match against. The current usecase is to match PBS datastores
based on properties such as if it is a removable datastore, if the
datastore has high usage or if the datastore is backed by S3.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 lib/pdm-api-types/src/resource.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/lib/pdm-api-types/src/resource.rs b/lib/pdm-api-types/src/resource.rs
index 7f74bf5..390e697 100644
--- a/lib/pdm-api-types/src/resource.rs
+++ b/lib/pdm-api-types/src/resource.rs
@@ -107,6 +107,22 @@ impl Resource {
             }
         }
     }
+
+    pub fn properties(&self) -> String {
+        let mut properties = Vec::new();
+        if let Resource::PbsDatastore(r) = self {
+            if let Some(backend_type) = &r.backend_type {
+                properties.push(backend_type.to_string());
+            }
+            if r.backing_device.is_some() {
+                properties.push("removable".to_string());
+            }
+            if r.usage > PBS_DATASTORE_HIGH_USAGE_THRESHOLD {
+                properties.push("high-usage".to_string());
+            }
+        }
+        properties.join(",")
+    }
 }
 
 #[api]
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 18/19] server: resources: add property matching for resources

[Christian Ebner]

Extends the search categories by the Property variant, which allows
to match against a list of properties defined on a resource.
Resources provide the list of properties as comma separates strings.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 server/src/api/resources.rs | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 3b629d1..57563b3 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -59,6 +59,7 @@ enum MatchCategory {
     Template,
     Remote,
     RemoteType,
+    Property,
 }
 
 impl std::str::FromStr for MatchCategory {
@@ -73,6 +74,7 @@ impl std::str::FromStr for MatchCategory {
             "template" => MatchCategory::Template,
             "remote" => MatchCategory::Remote,
             "remote-type" => MatchCategory::RemoteType,
+            "property" => MatchCategory::Property,
             _ => bail!("invalid category"),
         };
         Ok(category)
@@ -99,6 +101,10 @@ impl MatchCategory {
                 (Ok(a), Ok(b)) => a == b,
                 _ => false,
             },
+            MatchCategory::Property => value
+                .to_lowercase()
+                .split(",")
+                .any(|property| property == search_term.to_lowercase()),
         }
     }
 }
@@ -115,6 +121,7 @@ fn resource_matches_search_term(
             MatchCategory::Name => category.matches(resource.name(), &term.value),
             MatchCategory::Id => category.matches(&resource.id(), &term.value),
             MatchCategory::Status => category.matches(resource.status(), &term.value),
+            MatchCategory::Property => category.matches(&resource.properties(), &term.value),
             MatchCategory::Template => match resource {
                 Resource::PveQemu(PveQemuResource { template, .. })
                 | Resource::PveLxc(PveLxcResource { template, .. }) => {
@@ -151,6 +158,7 @@ fn remote_matches_search_term(
                 Some(false) => category.matches("offline", &term.value),
                 None => true,
             },
+            MatchCategory::Property => false,
             MatchCategory::Template => false,
             MatchCategory::RemoteType => category.matches(&remote.ty.to_string(), &term.value),
         },
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [PATCH datacenter-manager v3 19/19] ui: dashboard: add panel for PBS datastore statistics

[Christian Ebner]

Show a dashboard with the number of datastores in their respective
storage state.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- not present in previous version

 ui/src/dashboard/mod.rs                  |  76 +++-------
 ui/src/dashboard/pbs_datastores_panel.rs | 175 +++++++++++++++++++++++
 2 files changed, 193 insertions(+), 58 deletions(-)
 create mode 100644 ui/src/dashboard/pbs_datastores_panel.rs

diff --git a/ui/src/dashboard/mod.rs b/ui/src/dashboard/mod.rs
index d3da03d..07d5cd9 100644
--- a/ui/src/dashboard/mod.rs
+++ b/ui/src/dashboard/mod.rs
@@ -54,6 +54,9 @@ use status_row::DashboardStatusRow;
 
 mod filtered_tasks;
 
+mod pbs_datastores_panel;
+use pbs_datastores_panel::PbsDatastoresPanel;
+
 mod tasks;
 use tasks::TaskSummary;
 
@@ -282,6 +285,20 @@ impl PdmDashboard {
             )
     }
 
+    fn create_pbs_datastores_panel(&self) -> Panel {
+        let pbs_datastores = self
+            .status
+            .as_ref()
+            .map(|status| status.pbs_datastores.clone());
+
+        Panel::new()
+            .flex(1.0)
+            .width(300)
+            .title(self.create_title_with_icon("database", tr!("Backup Server Datastores")))
+            .border(true)
+            .with_child(PbsDatastoresPanel::new(pbs_datastores))
+    }
+
     fn reload(&mut self, ctx: &yew::Context<Self>) {
         let max_age = if self.loaded_once {
             self.config.max_age.unwrap_or(DEFAULT_MAX_AGE_S)
@@ -520,64 +537,7 @@ impl Component for PdmDashboard {
                         tr!("Backup Server Nodes"),
                         RemoteType::Pbs,
                     ))
-                    // FIXME: add further PBS support
-                    //.with_child(
-                    //    Panel::new()
-                    //        .flex(1.0)
-                    //        .width(300)
-                    //        .title(self.create_title_with_icon(
-                    //            "floppy-o",
-                    //            tr!("Backup Server Datastores"),
-                    //        ))
-                    //        .border(true)
-                    //        .with_child(if self.loading {
-                    //            Column::new()
-                    //                .padding(4)
-                    //                .class(FlexFit)
-                    //                .class(JustifyContent::Center)
-                    //                .class(AlignItems::Center)
-                    //                .with_child(html! {<i class={"pwt-loading-icon"} />})
-                    //        } else {
-                    //            Column::new()
-                    //                .padding(4)
-                    //                .class(FlexFit)
-                    //                .class(JustifyContent::Center)
-                    //                .gap(2)
-                    //                // FIXME: show more detailed status (usage?)
-                    //                .with_child(
-                    //                    Row::new()
-                    //                        .gap(2)
-                    //                        .with_child(
-                    //                            StorageState::Available.to_fa_icon().fixed_width(),
-                    //                        )
-                    //                        .with_child(tr!("available"))
-                    //                        .with_flex_spacer()
-                    //                        .with_child(
-                    //                            Container::from_tag("span").with_child(
-                    //                                self.status.pbs_datastores.available,
-                    //                            ),
-                    //                        ),
-                    //                )
-                    //                .with_optional_child(
-                    //                    (self.status.pbs_datastores.unknown > 0).then_some(
-                    //                        Row::new()
-                    //                            .gap(2)
-                    //                            .with_child(
-                    //                                StorageState::Unknown
-                    //                                    .to_fa_icon()
-                    //                                    .fixed_width(),
-                    //                            )
-                    //                            .with_child(tr!("unknown"))
-                    //                            .with_flex_spacer()
-                    //                            .with_child(
-                    //                                Container::from_tag("span").with_child(
-                    //                                    self.status.pbs_datastores.unknown,
-                    //                                ),
-                    //                            ),
-                    //                    ),
-                    //                )
-                    //        }),
-                    //)
+                    .with_child(self.create_pbs_datastores_panel())
                     .with_child(SubscriptionInfo::new()),
             )
             .with_child(
diff --git a/ui/src/dashboard/pbs_datastores_panel.rs b/ui/src/dashboard/pbs_datastores_panel.rs
new file mode 100644
index 0000000..e028476
--- /dev/null
+++ b/ui/src/dashboard/pbs_datastores_panel.rs
@@ -0,0 +1,175 @@
+use std::rc::Rc;
+
+use pdm_api_types::resource::{PbsDatastoreStatusCount, ResourceType};
+use pdm_search::{Search, SearchTerm};
+use proxmox_yew_comp::Status;
+use pwt::{
+    css::{self, TextAlign},
+    prelude::*,
+    widget::{Container, Fa, List, ListTile},
+};
+use yew::{
+    virtual_dom::{VComp, VNode},
+    Properties,
+};
+
+use crate::search_provider::get_search_provider;
+
+use super::loading_column;
+
+#[derive(PartialEq, Clone, Properties)]
+pub struct PbsDatastoresPanel {
+    status: Option<PbsDatastoreStatusCount>,
+}
+
+impl PbsDatastoresPanel {
+    /// Create new datastore status panel with given storage status counts
+    pub fn new(status: Option<PbsDatastoreStatusCount>) -> Self {
+        yew::props!(Self { status })
+    }
+}
+
+impl From<PbsDatastoresPanel> for VNode {
+    fn from(value: PbsDatastoresPanel) -> Self {
+        let comp = VComp::new::<PbsDatastoresPanelComponent>(Rc::new(value), None);
+        VNode::from(comp)
+    }
+}
+
+#[derive(PartialEq, Clone)]
+pub enum StatusRow {
+    Online(u64),
+    InMaintenance(u64),
+    Removable(u64),
+    S3Backend(u64),
+    HighUsage(u64),
+    Unknown(u64),
+    All(u64),
+}
+
+pub struct PbsDatastoresPanelComponent {}
+
+impl yew::Component for PbsDatastoresPanelComponent {
+    type Message = Search;
+    type Properties = PbsDatastoresPanel;
+
+    fn create(_ctx: &yew::Context<Self>) -> Self {
+        Self {}
+    }
+
+    fn update(&mut self, ctx: &Context<Self>, msg: Self::Message) -> bool {
+        if let Some(provider) = get_search_provider(ctx) {
+            provider.search(msg);
+        }
+
+        false
+    }
+
+    fn view(&self, ctx: &yew::Context<Self>) -> yew::Html {
+        let props = ctx.props();
+
+        let Some(status) = &props.status else {
+            return loading_column().into();
+        };
+
+        let data = vec![
+            StatusRow::Online(status.online),
+            StatusRow::InMaintenance(status.in_maintenance.unwrap_or_default()),
+            StatusRow::Removable(status.removable.unwrap_or_default()),
+            StatusRow::S3Backend(status.s3_backend.unwrap_or_default()),
+            StatusRow::HighUsage(status.high_usage.unwrap_or_default()),
+            StatusRow::Unknown(status.unknown.unwrap_or_default()),
+            StatusRow::All(status.online + status.in_maintenance.unwrap_or_default()),
+        ];
+
+        let tiles: Vec<_> = data
+            .into_iter()
+            .filter_map(|row| create_list_tile(ctx.link(), row))
+            .collect();
+
+        let list = List::new(tiles.len() as u64, move |idx: u64| {
+            tiles[idx as usize].clone()
+        })
+        .padding(4)
+        .class(css::Flex::Fill)
+        .grid_template_columns("auto auto 1fr auto");
+
+        list.into()
+    }
+}
+
+fn create_list_tile(
+    link: &html::Scope<PbsDatastoresPanelComponent>,
+    status_row: StatusRow,
+) -> Option<ListTile> {
+    let (icon, count, name, search_term) = match status_row {
+        StatusRow::Online(count) => (
+            Fa::from(Status::Success),
+            count,
+            "Online",
+            Some(("online", "status")),
+        ),
+        StatusRow::HighUsage(count) => (
+            Fa::from(Status::Warning),
+            count,
+            "High usage",
+            Some(("high-usage", "property")),
+        ),
+        StatusRow::InMaintenance(count) => (
+            Fa::new("wrench"),
+            count,
+            "In Maintenance",
+            Some(("in-maintenance", "status")),
+        ),
+        StatusRow::Removable(count) => (
+            Fa::new("plug"),
+            count,
+            "Removable",
+            Some(("removable", "property")),
+        ),
+        StatusRow::S3Backend(count) => (
+            Fa::new("cloud-upload"),
+            count,
+            "S3",
+            Some(("s3", "property")),
+        ),
+        StatusRow::Unknown(count) => (
+            Fa::from(Status::Unknown),
+            count,
+            "Unknown",
+            Some(("unknown", "property")),
+        ),
+        StatusRow::All(count) => (Fa::new("database"), count, "All", None),
+    };
+
+    Some(
+        ListTile::new()
+            .tabindex(0)
+            .interactive(true)
+            .with_child(icon)
+            .with_child(Container::new().padding_x(2).with_child(name))
+            .with_child(
+                Container::new()
+                    .class(TextAlign::Right)
+                    .padding_end(2)
+                    .with_child(count),
+            )
+            .with_child(Fa::new("search"))
+            .onclick(link.callback(move |_| create_pbs_datastores_status_search_term(search_term)))
+            .onkeydown(link.batch_callback(
+                move |event: KeyboardEvent| match event.key().as_str() {
+                    "Enter" | " " => Some(create_pbs_datastores_status_search_term(search_term)),
+                    _ => None,
+                },
+            )),
+    )
+}
+
+fn create_pbs_datastores_status_search_term(search_term: Option<(&str, &str)>) -> Search {
+    let resource_type: ResourceType = ResourceType::PbsDatastore;
+    let mut terms = vec![SearchTerm::new(resource_type.as_str()).category(Some("type"))];
+    if let Some((search_term, category)) = search_term {
+        terms.push(SearchTerm::new(search_term).category(Some(category)));
+    }
+    Search::with_terms(terms)
+}
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel