[pdm-devel] [RFC v2 datacenter-manager 0/3] PDM subscription status

[pdm-devel] [RFC v2 datacenter-manager 0/3] PDM subscription status

[Fabian Grünbichler]

this RFC series adds subscription endpoints and enterprise APT repo
handling for PDM itself.. there's considerable overlap with Dominik's
series[0], which is why I sent this now as RFC so we can discuss which
approach to take forward and merge the two series.

the patches here are largely untested, other than returning the right
error message when querying a system with not enough valid
subscriptions..

v2: resent with some smaller fixes

Fabian Grünbichler (3):
  subscription: add serverid field to node subscription info
  api types: add new SubscriptionStatistics
  api: add subscription endpoints

 lib/pdm-api-types/src/subscription.rs |  17 +++
 server/src/api/nodes/mod.rs           |   2 +
 server/src/api/nodes/subscription.rs  | 194 ++++++++++++++++++++++++++
 server/src/api/resources.rs           |   4 +-
 4 files changed, 216 insertions(+), 1 deletion(-)
 create mode 100644 server/src/api/nodes/subscription.rs

-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [RFC v2 datacenter-manager 1/3] subscription: add serverid field to node subscription info

[Fabian Grünbichler]

needed to re-use it internally for enterprise repository access.

intentionally not serialized to avoid leaking it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 lib/pdm-api-types/src/subscription.rs | 4 ++++
 server/src/api/resources.rs           | 2 ++
 2 files changed, 6 insertions(+)

diff --git a/lib/pdm-api-types/src/subscription.rs b/lib/pdm-api-types/src/subscription.rs
index 35910bc..64167cb 100644
--- a/lib/pdm-api-types/src/subscription.rs
+++ b/lib/pdm-api-types/src/subscription.rs
@@ -110,6 +110,10 @@ pub struct NodeSubscriptionInfo {
 
     /// The subscription level of the node
     pub level: SubscriptionLevel,
+
+    /// Serverid of the node, if accessible
+    #[serde(skip_serializing)]
+    pub serverid: Option<String>,
 }
 
 #[api(
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 55056e1..4beaa54 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -891,6 +891,7 @@ async fn fetch_remote_subscription_info(
                             status,
                             sockets: info.sockets,
                             key: info.key,
+                            serverid: info.serverid,
                             level: info
                                 .level
                                 .and_then(|level| level.parse().ok())
@@ -910,6 +911,7 @@ async fn fetch_remote_subscription_info(
                     sockets: None,
                     key: info.key,
                     level,
+                    serverid: info.serverid,
                 }
             });
 
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [RFC v2 datacenter-manager 2/3] api types: add new SubscriptionStatistics

[Fabian Grünbichler]

this is a different view for the global subscription status - instead of
looking at the lowest level per remote, count all nodes.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 lib/pdm-api-types/src/subscription.rs | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/lib/pdm-api-types/src/subscription.rs b/lib/pdm-api-types/src/subscription.rs
index 64167cb..d38611e 100644
--- a/lib/pdm-api-types/src/subscription.rs
+++ b/lib/pdm-api-types/src/subscription.rs
@@ -143,3 +143,16 @@ pub struct RemoteSubscriptions {
 
     pub state: RemoteSubscriptionState,
 }
+
+#[api]
+#[derive(Default, Serialize, Deserialize, Clone, PartialEq)]
+#[serde(rename_all = "kebab-case")]
+/// Lists the subscription level per node for the remote
+pub struct SubscriptionStatistics {
+    /// Total number of nodes across all remotes
+    pub total_nodes: usize,
+    /// Total number of active subscriptions across all remotes
+    pub active_subscriptions: usize,
+    /// Total number of community level subscriptions across all remotes
+    pub community: usize,
+}
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel

[pdm-devel] [RFC v2 datacenter-manager 3/3] api: add subscription endpoints

[Fabian Grünbichler]

for the PDM system itself, by proxy of how many of the remote nodes have valid
subscriptions above a certain level.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
v2: fix typo, adapt APT handling to only allow Basic+ PVE/PBS
subscriptions

 server/src/api/nodes/mod.rs          |   2 +
 server/src/api/nodes/subscription.rs | 194 +++++++++++++++++++++++++++
 server/src/api/resources.rs          |   2 +-
 3 files changed, 197 insertions(+), 1 deletion(-)
 create mode 100644 server/src/api/nodes/subscription.rs

diff --git a/server/src/api/nodes/mod.rs b/server/src/api/nodes/mod.rs
index a0fe14a..3a795d6 100644
--- a/server/src/api/nodes/mod.rs
+++ b/server/src/api/nodes/mod.rs
@@ -11,6 +11,7 @@ pub mod journal;
 pub mod network;
 pub mod rrddata;
 pub mod status;
+pub mod subscription;
 pub mod syslog;
 pub mod tasks;
 pub mod termproxy;
@@ -45,6 +46,7 @@ pub const SUBDIRS: SubdirMap = &sorted!([
     ("journal", &journal::ROUTER),
     ("network", &network::ROUTER),
     ("rrdata", &rrddata::ROUTER),
+    ("subscription", &subscription::ROUTER),
     ("status", &status::ROUTER),
     ("syslog", &syslog::ROUTER),
     ("tasks", &tasks::ROUTER),
diff --git a/server/src/api/nodes/subscription.rs b/server/src/api/nodes/subscription.rs
new file mode 100644
index 0000000..d9b44d5
--- /dev/null
+++ b/server/src/api/nodes/subscription.rs
@@ -0,0 +1,194 @@
+use std::collections::HashMap;
+
+use anyhow::{bail, Error};
+
+use proxmox_router::{Permission, Router};
+use proxmox_schema::api;
+use proxmox_schema::api_types::NODE_SCHEMA;
+use proxmox_subscription::files::update_apt_auth;
+use proxmox_subscription::{SubscriptionInfo, SubscriptionStatus};
+use proxmox_sys::fs::CreateOptions;
+
+use pdm_api_types::remotes::RemoteType;
+use pdm_api_types::subscription::{
+    NodeSubscriptionInfo, SubscriptionLevel, SubscriptionStatistics,
+};
+use pdm_api_types::PRIV_SYS_MODIFY;
+
+use crate::api::resources::get_subscription_info_for_remote;
+
+const PRODUCT_URL: &str = "https://www.proxmox.com/en/proxmox-datacenter-manager/pricing";
+const APT_AUTH_FN: &str = "/etc/apt/auth.conf.d/pdm.conf";
+const APT_AUTH_URL: &str = "enterprise.proxmox.com/debian/pdm";
+
+// minimum ratio of nodes with active subscriptions
+const SUBSCRIPTION_THRESHOLD: f64 = 0.9;
+// max ratio of nodes with community subscriptions, among nodes with subscriptions
+const COMMUNITY_THRESHOLD: f64 = 0.4;
+
+fn apt_auth_file_opts() -> CreateOptions {
+    let mode = nix::sys::stat::Mode::from_bits_truncate(0o0600);
+    CreateOptions::new().perm(mode).owner(nix::unistd::ROOT)
+}
+
+async fn get_all_subscription_infos(
+) -> Result<HashMap<String, (RemoteType, HashMap<String, Option<NodeSubscriptionInfo>>)>, Error> {
+    let (remotes_config, _digest) = pdm_config::remotes::config()?;
+
+    let mut subscription_info = HashMap::new();
+    for (remote_name, remote) in remotes_config.iter() {
+        match get_subscription_info_for_remote(remote, 24 * 60 * 60).await {
+            Ok(info) => {
+                subscription_info.insert(remote_name.to_string(), (remote.ty, info));
+            }
+            Err(err) => {
+                log::debug!("Failed to get subscription info for remote {remote_name} - {err}");
+                subscription_info.insert(remote_name.to_string(), (remote.ty, HashMap::new()));
+            }
+        }
+    }
+    Ok(subscription_info)
+}
+
+fn count_subscriptions(
+    subscription_infos: &HashMap<
+        String,
+        (RemoteType, HashMap<String, Option<NodeSubscriptionInfo>>),
+    >,
+) -> SubscriptionStatistics {
+    let mut stats = SubscriptionStatistics::default();
+    for (_remote, (_remote_type, remote_infos)) in subscription_infos.iter() {
+        if remote_infos.is_empty() {
+            // count remotes without info as at least one node
+            stats.total_nodes += 1;
+            continue;
+        }
+        for (_node, node_info) in remote_infos.iter() {
+            stats.total_nodes += 1;
+            if let Some(info) = node_info {
+                if info.status == SubscriptionStatus::Active {
+                    stats.active_subscriptions += 1;
+                    if info.level == SubscriptionLevel::Community {
+                        stats.community += 1;
+                    }
+                }
+            }
+        }
+    }
+    stats
+}
+
+fn check_counts(stats: SubscriptionStatistics) -> Result<(), Error> {
+    let subscribed_ratio = stats.active_subscriptions as f64 / stats.total_nodes as f64;
+    let community_ratio = stats.community as f64 / stats.active_subscriptions as f64;
+
+    if subscribed_ratio > SUBSCRIPTION_THRESHOLD {
+        if community_ratio < COMMUNITY_THRESHOLD {
+            return Ok(());
+        } else {
+            bail!("Too many remote nodes with community level subscription!");
+        }
+    } else {
+        bail!("Too many remote nodes without active subscription!");
+    }
+}
+
+#[api(
+    access: { permission: &Permission::Anybody, },
+    input: {
+        properties: {
+            node: {
+                schema: NODE_SCHEMA,
+            },
+        },
+    },
+    returns: {
+        type: SubscriptionInfo,
+    }
+)]
+/// Return subscription status
+pub async fn get_subscription() -> Result<SubscriptionInfo, Error> {
+    let infos = get_all_subscription_infos().await?;
+
+    let stats = count_subscriptions(&infos);
+
+    if let Err(err) = check_counts(stats) {
+        Ok(SubscriptionInfo {
+            status: SubscriptionStatus::Invalid,
+            message: Some(format!("{err}")),
+            serverid: None,
+            url: Some(PRODUCT_URL.into()),
+            ..Default::default()
+        })
+    } else {
+        Ok(SubscriptionInfo {
+            status: SubscriptionStatus::Active,
+            url: Some(PRODUCT_URL.into()),
+            ..Default::default()
+        })
+    }
+}
+
+#[api(
+    input: {
+        properties: {
+            node: {
+                schema: NODE_SCHEMA,
+            },
+        },
+    },
+    protected: true,
+    access: {
+        permission: &Permission::Privilege(&["system"], PRIV_SYS_MODIFY, false),
+    },
+)]
+/// Update subscription information
+pub async fn check_subscription() -> Result<(), Error> {
+    let infos = get_all_subscription_infos().await?;
+    let stats = count_subscriptions(&infos);
+
+    if let Err(err) = check_counts(stats) {
+        update_apt_auth(APT_AUTH_FN, apt_auth_file_opts(), APT_AUTH_URL, None, None)?;
+        return Err(err);
+    }
+
+    let mut found = false;
+    'outer: for (remote, (remote_type, remote_info)) in infos.iter() {
+        if *remote_type != RemoteType::Pve || *remote_type != RemoteType::Pbs {
+            continue;
+        }
+        for (node, node_info) in remote_info.iter() {
+            if let Some(info) = node_info {
+                if info.status == SubscriptionStatus::Active
+                    && info.level >= SubscriptionLevel::Basic
+                    && info.key.is_some()
+                    && info.serverid.is_some()
+                {
+                    log::info!("Using subscription of node '{node}' of remote '{remote}' for enterprise repository access");
+                    update_apt_auth(
+                        APT_AUTH_FN,
+                        apt_auth_file_opts(),
+                        APT_AUTH_URL,
+                        info.key.clone(),
+                        info.serverid.clone(),
+                    )?;
+                    found = true;
+                    break 'outer;
+                }
+            }
+        }
+    }
+
+    if !found {
+        log::warn!(
+            "No valid Basic+ subscription found for configuring enterprise repository access.."
+        );
+        update_apt_auth(APT_AUTH_FN, apt_auth_file_opts(), APT_AUTH_URL, None, None)?;
+    }
+
+    Ok(())
+}
+
+pub const ROUTER: Router = Router::new()
+    .get(&API_METHOD_GET_SUBSCRIPTION)
+    .post(&API_METHOD_CHECK_SUBSCRIPTION);
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 4beaa54..adab021 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -767,7 +767,7 @@ static SUBSCRIPTION_CACHE: LazyLock<RwLock<HashMap<String, CachedSubscriptionSta
 ///
 /// If recent enough cached data is available, it is returned
 /// instead of calling out to the remote.
-async fn get_subscription_info_for_remote(
+pub async fn get_subscription_info_for_remote(
     remote: &Remote,
     max_age: u64,
 ) -> Result<HashMap<String, Option<NodeSubscriptionInfo>>, Error> {
-- 
2.47.3



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel