From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pdm-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 7A6EB1FF189
for <inbox@lore.proxmox.com>; Tue, 18 Feb 2025 16:26:46 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 9244910B59;
Tue, 18 Feb 2025 16:26:43 +0100 (CET)
Date: Tue, 18 Feb 2025 16:26:09 +0100
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: Lukas Wagner <l.wagner@proxmox.com>
Message-ID: <aguxeubkyqnqrfsrmmo4suxmap34ep34bhs5l3rhhqrjr33hlo@vw7u7jhojkka>
References: <20250214130653.283012-1-l.wagner@proxmox.com>
<20250214130653.283012-4-l.wagner@proxmox.com>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <20250214130653.283012-4-l.wagner@proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.080 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [lib.rs]
Subject: Re: [pdm-devel] [PATCH proxmox-datacenter-manager v2 03/28]
pdm-api-types: add CollectionSettings type
X-BeenThere: pdm-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Datacenter Manager development discussion
<pdm-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pdm-devel>,
<mailto:pdm-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pdm-devel/>
List-Post: <mailto:pdm-devel@lists.proxmox.com>
List-Help: <mailto:pdm-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel>,
<mailto:pdm-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Datacenter Manager development discussion
<pdm-devel@lists.proxmox.com>
Cc: pdm-devel@lists.proxmox.com
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pdm-devel-bounces@lists.proxmox.com
Sender: "pdm-devel" <pdm-devel-bounces@lists.proxmox.com>
On Fri, Feb 14, 2025 at 02:06:28PM +0100, Lukas Wagner wrote:
> This commit adds the CollectionSettings type which holds settings for
> the metric collection system. Included are collection interval, max
> concurrency and upper/lower bounds for the metric collection loop.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> lib/pdm-api-types/src/lib.rs | 3 +
> lib/pdm-api-types/src/metric_collection.rs | 188 +++++++++++++++++++++
> 2 files changed, 191 insertions(+)
> create mode 100644 lib/pdm-api-types/src/metric_collection.rs
>
> diff --git a/lib/pdm-api-types/src/lib.rs b/lib/pdm-api-types/src/lib.rs
> index 38449071..6115e41c 100644
> --- a/lib/pdm-api-types/src/lib.rs
> +++ b/lib/pdm-api-types/src/lib.rs
> @@ -19,6 +19,9 @@ pub use acl::*;
> mod node_config;
> pub use node_config::*;
>
> +mod metric_collection;
> +pub use metric_collection::*;
> +
> mod proxy;
> pub use proxy::HTTP_PROXY_SCHEMA;
>
> diff --git a/lib/pdm-api-types/src/metric_collection.rs b/lib/pdm-api-types/src/metric_collection.rs
> new file mode 100644
> index 00000000..92487d6c
> --- /dev/null
> +++ b/lib/pdm-api-types/src/metric_collection.rs
> @@ -0,0 +1,188 @@
> +//! API types for metric collection settings.
> +
> +use serde::{Deserialize, Serialize};
> +
> +use proxmox_schema::{api, Updater};
> +
> +/// Default metric collection interval.
> +pub const DEFAULT_COLLECTION_INTERVAL: u64 = 600;
> +
> +/// Minimum metric collection interval.
> +pub const MIN_COLLECTION_INTERVAL: u64 = 10;
> +
> +/// Maximum metric collection interval.
> +/// PVE and PBS keep 30 minutes of metric history,
> +/// maximum is set to 25 minutes to leave some headroom.
> +pub const MAX_COLLECTION_INTERVAL: u64 = 1500;
> +
> +/// Default number of concurrent connections.
> +pub const DEFAULT_CONCURRENCY: usize = 10;
> +/// Maximum number of concurrent connections.
> +pub const MAX_CONCURRENCY: u64 = 50;
> +/// Minimum number of concurrent connections (no concurrency).
> +pub const MIN_CONCURRENCY: u64 = 1;
> +
> +/// Default upper bound for the random delay for the
> +/// main metric collection loop.
> +pub const DEFAULT_MAX_OFFSET: u64 = 10;
> +/// Default lower bound for the random delay for the
> +/// main metric collection loop.
> +pub const DEFAULT_MIN_OFFSET: u64 = 0;
> +
> +/// Highest configurable upper bound for the random interval offset for the main loop.
> +pub const MAX_INTERVAL_OFFSET: u64 = 300;
> +/// Lowest configureable lower bound for the random interval offset for the main loop.
> +pub const MIN_INTERVAL_OFFSET: u64 = 0;
> +
> +/// Default upper bound for the random individual connection delay.
> +pub const DEFAULT_MAX_CONNECTION_DELAY: u64 = 100;
> +/// Default lower bound for the random individual connection delay.
> +pub const DEFAULT_MIN_CONNECTION_DELAY: u64 = 0;
> +
> +/// Highest configurable upper bound for the random individual connection delay.
> +pub const MAX_CONNECTION_DELAY: u64 = 1000;
> +/// Lowest configurable lower bound for the random individual connection delay.
> +pub const MIN_CONNECTION_DELAY: u64 = 0;
> +
> +#[api(
> + properties: {
> + "collection-interval" : {
> + optional: true,
> + default: DEFAULT_COLLECTION_INTERVAL as isize,
> + minimum: MIN_COLLECTION_INTERVAL as isize,
> + maximum: MAX_COLLECTION_INTERVAL as isize,
I thought about this the first time around but then forgot again:
Given that it is possible we might change these types (see Stefan's
patch for proxmox-schema) and because it is much more convenient
anyway you could use `as _` for these casts.
While I'm usually not a fan of `_` casts - I think this might be okay
for schema declarations.
(No need for a v3 if there's nothing else...)
> + },
> + "max-concurrent-connections" : {
> + optional: true,
> + default: DEFAULT_CONCURRENCY as isize,
> + minimum: MIN_CONCURRENCY as isize,
> + maximum: MAX_CONCURRENCY as isize,
> + },
> + "max-interval-offset" : {
> + optional: true,
> + default: DEFAULT_MAX_OFFSET as isize,
> + minimum: MIN_INTERVAL_OFFSET as isize,
> + maximum: MAX_INTERVAL_OFFSET as isize,
> + },
> + "min-interval-offset" : {
> + optional: true,
> + default: DEFAULT_MIN_OFFSET as isize,
> + minimum: MIN_INTERVAL_OFFSET as isize,
> + maximum: MAX_INTERVAL_OFFSET as isize,
> + },
> + "max-connection-delay" : {
> + optional: true,
> + default: DEFAULT_MAX_CONNECTION_DELAY as isize,
> + minimum: MIN_CONNECTION_DELAY as isize,
> + maximum: MAX_CONNECTION_DELAY as isize,
> + },
> + "min-connection-delay" : {
> + optional: true,
> + default: DEFAULT_MIN_CONNECTION_DELAY as isize,
> + minimum: MIN_CONNECTION_DELAY as isize,
> + maximum: MAX_CONNECTION_DELAY as isize,
> + },
> + },
> +)]
> +#[derive(Clone, Default, Deserialize, Serialize, Updater)]
> +#[serde(rename_all = "kebab-case")]
> +/// Settings for the metric collection system.
> +pub struct CollectionSettings {
> + /// Collection settings ID
> + #[updater(skip)]
> + pub id: String,
> +
> + /// Interval in seconds at which to collect metrics.
> + /// The point in time at which metrics are collected
> + /// are aligned based on the collection interval. For instance,
> + /// a collection interval of 300 (5 minutes) would schedule metric collection
> + /// at 11:00:00, 11:05:00 (without accounting for the random offset).
> + ///
> + /// To avoid load spikes, metric collection runs are offeset by
> + /// a random number of seconds between min_interval_offset and max_interval_offset.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub collection_interval: Option<u64>,
> +
> + /// Maximum number of concurrent connections while collecting metrics.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub max_concurrent_connections: Option<usize>,
> +
> + /// Maximum offset in seconds for the metric collection interval.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub max_interval_offset: Option<u64>,
> +
> + /// Minimum offset in seconds for the metric collection interval.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub min_interval_offset: Option<u64>,
> +
> + /// Maximum random delay in milliseconds for each outgoing connection.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub max_connection_delay: Option<u64>,
> +
> + /// Minimum random delay in milliseconds for each outgoing connection.
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub min_connection_delay: Option<u64>,
> +}
> +
> +#[api]
> +#[derive(Copy, Clone, Deserialize, Serialize)]
> +#[serde(rename_all = "kebab-case")]
> +/// Deletable property for [`CollectionSettings`].
> +pub enum DeletableCollectionSettingsProperty {
> + /// Delete 'collection-interval'.
> + CollectionInterval,
> + /// Delete 'max-concurrent-connections'.
> + MaxConcurrentConnections,
> + /// Delete 'max-interval-offset'.
> + MaxIntervalOffset,
> + /// Delete 'min-interval-offset'.
> + MinIntervalOffset,
> + /// Delete 'min-connection-delay'.
> + MaxConnectionDelay,
> + /// Delete 'min-connection-delay'.
> + MinConnectionDelay,
> +}
> +
> +impl CollectionSettings {
> + /// Create a new settings instance with a given `id`.
> + pub fn new(id: &str) -> Self {
> + Self {
> + id: id.into(),
> + ..Default::default()
> + }
> + }
> +
> + /// Return the collection interval or the default if not configured.
> + pub fn collection_interval_or_default(&self) -> u64 {
> + self.collection_interval
> + .unwrap_or(DEFAULT_COLLECTION_INTERVAL)
> + }
> +
> + /// Return the number of allowed concurrent connections or the default if not configured.
> + pub fn max_concurrent_connections_or_default(&self) -> usize {
> + self.max_concurrent_connections
> + .unwrap_or(DEFAULT_CONCURRENCY)
> + }
> +
> + /// Return the upper bound for the main loop delay or the default if not configured.
> + pub fn max_interval_offset_or_default(&self) -> u64 {
> + self.max_interval_offset.unwrap_or(DEFAULT_MAX_OFFSET)
> + }
> +
> + /// Return the lower bound for the main loop delay or the default if not configured.
> + pub fn min_interval_offset_or_default(&self) -> u64 {
> + self.min_interval_offset.unwrap_or(DEFAULT_MIN_OFFSET)
> + }
> +
> + /// Return the upper bound for the connection delay or the default if not configured.
> + pub fn max_connection_delay_or_default(&self) -> u64 {
> + self.max_connection_delay
> + .unwrap_or(DEFAULT_MAX_CONNECTION_DELAY)
> + }
> +
> + /// Return the lower bound for the connection delay or the default if not configured.
> + pub fn min_connection_delay_or_default(&self) -> u64 {
> + self.min_connection_delay
> + .unwrap_or(DEFAULT_MIN_CONNECTION_DELAY)
> + }
> +}
> --
> 2.39.5
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel