From: Lukas Wagner <l.wagner@proxmox.com>
To: pdm-devel@lists.proxmox.com
Subject: [PATCH datacenter-manager 5/5] parallel fetcher: add module documentation
Date: Wed, 4 Feb 2026 16:27:23 +0100 [thread overview]
Message-ID: <20260204152723.482258-6-l.wagner@proxmox.com> (raw)
In-Reply-To: <20260204152723.482258-1-l.wagner@proxmox.com>
Adding a (no-run) doctest for the module level documentation gives users
a quick idea on how to use this helper.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/parallel_fetcher.rs | 65 ++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/server/src/parallel_fetcher.rs b/server/src/parallel_fetcher.rs
index 57011096..0abd9a50 100644
--- a/server/src/parallel_fetcher.rs
+++ b/server/src/parallel_fetcher.rs
@@ -1,3 +1,66 @@
+//! Helpers that can be used to parallelize API requests to remotes.
+//!
+//! ```no_run
+//! # use anyhow::Error;
+//! #
+//! # use pdm_api_types::remotes::{RemoteType, Remote};
+//! # use server::parallel_fetcher::ParallelFetcher;
+//! #
+//! # #[tokio::main]
+//! # async fn main() -> Result<(), Error> {
+//! # let remotes: Vec<Remote> = Vec::new();
+//! #
+//! async fn fetch_meaning(
+//! _context: (),
+//! remote: Remote,
+//! node: String,
+//! ) -> Result<i32, Error> {
+//! match remote.ty {
+//! RemoteType::Pve => {
+//! // Perform the API request here and return some result.
+//! Ok(42)
+//! },
+//! RemoteType::Pbs => Ok(42),
+//! }
+//! }
+//!
+//! // This context can be passed to the function what is executed for every remote node.
+//! let context = ();
+//!
+//! let fetcher = ParallelFetcher::builder(context)
+//! .max_connections(10)
+//! .max_connections_per_remote(2)
+//! .build();
+//!
+//! let fetch_result = fetcher
+//! .do_for_all_remote_nodes(remotes.into_iter(), fetch_meaning)
+//! .await;
+//!
+//! for remote_outcome in fetch_result {
+//! match remote_outcome.nodes() {
+//! Ok(node_outcomes) => {
+//! for node_outcome in node_outcomes {
+//! match node_outcome.data() {
+//! Ok(meaning) => assert_eq!(*meaning, 42),
+//! Err(err) =>
+//! log::error!(
+//! "failed to retrieve result for node {}",
+//! node_outcome.node_name()
+//! ),
+//! }
+//! }
+//! }
+//! Err(err) => log::error!(
+//! "failed to connect to remote {}",
+//! remote_outcome.remote()
+//! ),
+//! }
+//! }
+//!
+//! # Ok(())
+//! # }
+//! ```
+
use std::fmt::Debug;
use std::future::Future;
use std::sync::Arc;
@@ -14,7 +77,9 @@ use pdm_api_types::remotes::{Remote, RemoteType};
use crate::connection;
+/// Maximum number of parallel outgoing API requests.
pub const DEFAULT_MAX_CONNECTIONS: usize = 20;
+/// Maximum number of parallel outgoing API requests to the *same* remote.
pub const DEFAULT_MAX_CONNECTIONS_PER_REMOTE: usize = 5;
/// Outcome type produced by [`ParallelFetcher::do_for_all_remotes`] or
--
2.47.3
prev parent reply other threads:[~2026-02-04 15:27 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-04 15:27 [PATCH datacenter-manager 0/5] improvements for ParallelFetcher Lukas Wagner
2026-02-04 15:27 ` [PATCH datacenter-manager 1/5] parallel fetcher: clean up imports Lukas Wagner
2026-02-04 15:27 ` [PATCH datacenter-manager 2/5] parallel fetcher: make sure to inherit log context Lukas Wagner
2026-02-04 15:27 ` [PATCH datacenter-manager 3/5] parallel fetcher: add builder and make struct members private Lukas Wagner
2026-02-04 15:27 ` [PATCH datacenter-manager 4/5] parallel fetcher: improve result type ergonomics Lukas Wagner
2026-02-04 15:27 ` Lukas Wagner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260204152723.482258-6-l.wagner@proxmox.com \
--to=l.wagner@proxmox.com \
--cc=pdm-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.