From: Lukas Wagner <l.wagner@proxmox.com>
To: pdm-devel@lists.proxmox.com
Subject: [PATCH datacenter-manager v2 2/2] parallel fetcher: add module documentation
Date: Fri, 6 Feb 2026 10:43:04 +0100 [thread overview]
Message-ID: <20260206094304.117465-3-l.wagner@proxmox.com> (raw)
In-Reply-To: <20260206094304.117465-1-l.wagner@proxmox.com>
Adding a (no-run) doctest for the module level documentation gives users
a quick idea on how to use this helper.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/parallel_fetcher.rs | 65 ++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/server/src/parallel_fetcher.rs b/server/src/parallel_fetcher.rs
index b6f49d6f..83f083ca 100644
--- a/server/src/parallel_fetcher.rs
+++ b/server/src/parallel_fetcher.rs
@@ -1,3 +1,66 @@
+//! Helpers that can be used to parallelize API requests to remotes.
+//!
+//! ```no_run
+//! # use anyhow::Error;
+//! #
+//! # use pdm_api_types::remotes::{RemoteType, Remote};
+//! # use server::parallel_fetcher::ParallelFetcher;
+//! #
+//! # #[tokio::main]
+//! # async fn main() -> Result<(), Error> {
+//! # let remotes: Vec<Remote> = Vec::new();
+//! #
+//! async fn fetch_meaning(
+//! _context: (),
+//! remote: Remote,
+//! node: String,
+//! ) -> Result<i32, Error> {
+//! match remote.ty {
+//! RemoteType::Pve => {
+//! // Perform the API request here and return some result.
+//! Ok(42)
+//! },
+//! RemoteType::Pbs => Ok(42),
+//! }
+//! }
+//!
+//! // This context can be passed to the function what is executed for every remote node.
+//! let context = ();
+//!
+//! let fetcher = ParallelFetcher::builder(context)
+//! .max_connections(10)
+//! .max_connections_per_remote(2)
+//! .build();
+//!
+//! let fetcher_response = fetcher
+//! .do_for_all_remote_nodes(remotes.into_iter(), fetch_meaning)
+//! .await;
+//!
+//! for remote_response in fetcher_response {
+//! match remote_response.nodes() {
+//! Ok(node_responses) => {
+//! for node_response in node_responses {
+//! match node_response.data() {
+//! Ok(meaning) => assert_eq!(*meaning, 42),
+//! Err(err) =>
+//! log::error!(
+//! "failed to retrieve result for node {}",
+//! node_response.node_name()
+//! ),
+//! }
+//! }
+//! }
+//! Err(err) => log::error!(
+//! "failed to connect to remote {}",
+//! remote_response.remote()
+//! ),
+//! }
+//! }
+//!
+//! # Ok(())
+//! # }
+//! ```
+
use std::fmt::Debug;
use std::future::Future;
use std::sync::Arc;
@@ -14,7 +77,9 @@ use pdm_api_types::remotes::{Remote, RemoteType};
use crate::connection;
+/// Maximum number of parallel outgoing API requests.
pub const DEFAULT_MAX_CONNECTIONS: usize = 20;
+/// Maximum number of parallel outgoing API requests to the *same* remote.
pub const DEFAULT_MAX_CONNECTIONS_PER_REMOTE: usize = 5;
/// Response container type produced by [`ParallelFetcher::do_for_all_remotes`] or
--
2.47.3
prev parent reply other threads:[~2026-02-06 9:42 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-06 9:43 [PATCH datacenter-manager v2 0/2] improvements for ParallelFetcher Lukas Wagner
2026-02-06 9:43 ` [PATCH datacenter-manager v2 1/2] parallel fetcher: improve result type ergonomics Lukas Wagner
2026-02-06 9:43 ` Lukas Wagner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260206094304.117465-3-l.wagner@proxmox.com \
--to=l.wagner@proxmox.com \
--cc=pdm-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.