[PATCH many v2 0/8] fix #7530: Implement download bandwidth and restore limits for pbs-restore

[PATCH many v2 0/8] fix #7530: Implement download bandwidth and restore limits for pbs-restore

[Christian Ebner]

This patch series implements rate limiting functionality for restores
using pbs-restore. In particular it exposes optional `bwlimit` and
`rate-limit` flags, the former to impose download bandwidth limits,
the latter to impose limits on the output data stream, taking into
account zero size chunks and data after decompression.

The output rate limit is further used to enforce the limit when
performing restores via the qemu-server stack, so it is imposed also
when the restore is performed via qmrestore and the PVE UI.

For proxmox-backup-qemu some cleanups and dependencies are bumped as
part of the series as well in order to keep it up-to-date and compile
it cleanly with the latest version available of the proxmox-backup
submodule.

Required build order is:
- pbs-api-types
- proxmox-backup-qemu
- pve-qemu
- qemu-server

Changes since version 1:
- Rebased all patches onto respective current master
- Refactored proxmox-backup-qemu patches as the submodule
  dependency has already been bumped since the previous version.

Link to the bugtracker issue:
https://bugzilla.proxmox.com/show_bug.cgi?id=7530


proxmox:

Christian Ebner (1):
  pbs-api-types: expose ClientRateLimitConfig fields as pub

 pbs-api-types/src/traffic_control.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


proxmox-backup-qemu:

Christian Ebner (5):
  commands/restore: fix useless borrow clippy warnings for archive names
  commands: fix clippy error for reimplementing Result::ok()
  commands: extend stricter type checks for guest config archive names
  restore: implement `bw-limit` parameter imposing download rate limits
  restore: implement `rate-limit` imposing chunk data stream limits

 Cargo.toml      |  4 +++-
 current-api.h   | 14 +++++++++++
 src/backup.rs   |  3 ++-
 src/commands.rs | 29 +++++++++++++----------
 src/lib.rs      | 63 +++++++++++++++++++++++++++++++++++++++++++++++--
 src/restore.rs  | 44 ++++++++++++++++++++++++++--------
 6 files changed, 130 insertions(+), 27 deletions(-)


pve-qemu:

Christian Ebner (1):
  fix #7530: pbs-restore: expose optional rate limit parameters

 ...se-optional-parameters-imposing-band.patch | 78 +++++++++++++++++++
 debian/patches/series                         |  1 +
 2 files changed, 79 insertions(+)
 create mode 100644 debian/patches/pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch


qemu-server:

Christian Ebner (1):
  pbs-restore: set restore bandwidth limit for volumes

 debian/control        | 2 +-
 src/PVE/QemuServer.pm | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)


Summary over all repositories:
  11 files changed, 215 insertions(+), 30 deletions(-)

-- 
Generated by murpp 0.11.0

[PATCH proxmox v2 1/8] pbs-api-types: expose ClientRateLimitConfig fields as pub

[Christian Ebner]

Required to construct the struct when not deserializing, as will
be used in the PBS client for pbs-restore.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-api-types/src/traffic_control.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pbs-api-types/src/traffic_control.rs b/pbs-api-types/src/traffic_control.rs
index d9ba5e54..34d7b176 100644
--- a/pbs-api-types/src/traffic_control.rs
+++ b/pbs-api-types/src/traffic_control.rs
@@ -96,9 +96,9 @@ const CLIENT_BURST_SCHEMA: Schema = HumanByte::API_SCHEMA
 /// Client Rate Limit Configuration
 pub struct ClientRateLimitConfig {
     #[serde(skip_serializing_if = "Option::is_none")]
-    rate: Option<HumanByte>,
+    pub rate: Option<HumanByte>,
     #[serde(skip_serializing_if = "Option::is_none")]
-    burst: Option<HumanByte>,
+    pub burst: Option<HumanByte>,
 }
 
 #[api(
-- 
2.47.3

[PATCH proxmox-backup-qemu v2 2/8] commands/restore: fix useless borrow clippy warnings for archive names

[Christian Ebner]

The archive name variable for given callsides already hold a shared
reference only, no need to borrow again.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/commands.rs |  2 +-
 src/restore.rs  | 12 ++++--------
 2 files changed, 5 insertions(+), 9 deletions(-)

diff --git a/src/commands.rs b/src/commands.rs
index 89ed69c..95e7c17 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -484,7 +484,7 @@ pub(crate) async fn finish_backup(
             ..UploadOptions::default()
         };
         let stats = client
-            .upload_blob_from_data(rsa_encrypted_key, &target, options)
+            .upload_blob_from_data(rsa_encrypted_key, target, options)
             .await?;
         manifest
             .lock()
diff --git a/src/restore.rs b/src/restore.rs
index 75d0a7a..8f518bc 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -151,9 +151,7 @@ impl RestoreTask {
             None => bail!("no manifest"),
         };
 
-        let index = client
-            .download_fixed_index(&manifest, &archive_name)
-            .await?;
+        let index = client.download_fixed_index(&manifest, archive_name).await?;
 
         let (_, zero_chunk_digest) = DataChunkBuilder::build_zero_chunk(
             self.crypt_config.as_ref().map(Arc::as_ref),
@@ -163,7 +161,7 @@ impl RestoreTask {
 
         let most_used = index.find_most_used_chunks(8);
 
-        let file_info = manifest.lookup_file_info(&archive_name)?;
+        let file_info = manifest.lookup_file_info(archive_name)?;
 
         let chunk_reader = RemoteChunkReader::new(
             Arc::clone(&client),
@@ -274,13 +272,11 @@ impl RestoreTask {
             None => bail!("no manifest"),
         };
 
-        let index = client
-            .download_fixed_index(&manifest, &archive_name)
-            .await?;
+        let index = client.download_fixed_index(&manifest, archive_name).await?;
         let archive_size = index.index_bytes();
         let most_used = index.find_most_used_chunks(8);
 
-        let file_info = manifest.lookup_file_info(&archive_name)?;
+        let file_info = manifest.lookup_file_info(archive_name)?;
 
         let chunk_reader = RemoteChunkReader::new(
             Arc::clone(&client),
-- 
2.47.3

[PATCH proxmox-backup-qemu v2 3/8] commands: fix clippy error for reimplementing Result::ok()

[Christian Ebner]

Call Result::ok() on the index download result, avoiding
reimplementation thereof.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/commands.rs | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/src/commands.rs b/src/commands.rs
index 95e7c17..d482fd5 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -195,14 +195,11 @@ pub(crate) async fn register_image(
 
     let index = match manifest {
         Some(manifest) => {
-            match client
+            // not having a previous index is not fatal, so ignore errors
+            client
                 .download_previous_fixed_index(&archive_name, &manifest, Arc::clone(&known_chunks))
                 .await
-            {
-                Ok(index) => Some(index),
-                // not having a previous index is not fatal, so ignore errors
-                Err(_) => None,
-            }
+                .ok()
         }
         None => None,
     };
-- 
2.47.3

[PATCH proxmox-backup-qemu v2 4/8] commands: extend stricter type checks for guest config archive names

[Christian Ebner]

Further extends the stricter BackupArchiveName type as introduced in
[0] to also apply to internal add_config() helper for guest config.

Due to the stricter checks, add_config() now allows the server side
extensions as defined in the type for blobs and auto-expands the
server side `.blob` extension when required, the archive type
is explicitley checked instead.

While at it, make sure to always use the latest stricter check by
depending on version 1.0.20, which is not strictly required here but
the submodule already depends on it as well and it protects against
building future versions with the still more relaxed archive name
parsing.

The api type changes affect internal API's only, public API's types
remain unchanged.

[0] https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=b57274a5a64791e7245bc0655ce0f133668db145

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml      |  2 +-
 src/backup.rs   |  3 ++-
 src/commands.rs | 18 ++++++++++++------
 3 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index c50a0a2..d73324f 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -37,7 +37,7 @@ proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
 proxmox-sys = "1"
 
-pbs-api-types  = { version = "1" }
+pbs-api-types  = { version = "1.0.20" }
 
 pbs-client     = { path = "submodules/proxmox-backup/pbs-client" }
 pbs-datastore  = { path = "submodules/proxmox-backup/pbs-datastore" }
diff --git a/src/backup.rs b/src/backup.rs
index 61ebcfb..3a591ea 100644
--- a/src/backup.rs
+++ b/src/backup.rs
@@ -10,7 +10,7 @@ use tokio::runtime::Runtime;
 use proxmox_async::runtime::get_runtime_with_builder;
 use proxmox_sys::fs::file_get_contents;
 
-use pbs_api_types::{BackupType, CryptMode};
+use pbs_api_types::{BackupArchiveName, BackupType, CryptMode};
 use pbs_client::{BackupWriter, BackupWriterOptions, HttpClient, HttpClientOptions};
 use pbs_datastore::BackupManifest;
 use pbs_key_config::{load_and_decrypt_key, rsa_encrypt_key_config, KeyConfig};
@@ -186,6 +186,7 @@ impl BackupTask {
 
     pub async fn add_config(&self, name: String, data: Vec<u8>) -> Result<c_int, Error> {
         self.check_aborted()?;
+        let name: BackupArchiveName = name.parse()?;
 
         let command_future = add_config(
             self.need_writer()?,
diff --git a/src/commands.rs b/src/commands.rs
index d482fd5..1b009d5 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -6,7 +6,9 @@ use std::sync::{Arc, Mutex};
 use futures::future::{Future, TryFutureExt};
 use serde_json::json;
 
-use pbs_api_types::{BackupArchiveName, CryptMode, ENCRYPTED_KEY_BLOB_NAME, MANIFEST_BLOB_NAME};
+use pbs_api_types::{
+    ArchiveType, BackupArchiveName, CryptMode, ENCRYPTED_KEY_BLOB_NAME, MANIFEST_BLOB_NAME,
+};
 use pbs_client::{BackupWriter, H2Client, UploadOptions};
 use pbs_datastore::data_blob::DataChunkBuilder;
 use pbs_datastore::index::IndexFile;
@@ -94,14 +96,19 @@ async fn register_zero_chunk(
 pub(crate) async fn add_config(
     client: Arc<BackupWriter>,
     manifest: Arc<Mutex<BackupManifest>>,
-    name: String,
+    config_blob_name: BackupArchiveName,
     data: Vec<u8>,
     compress: bool,
     crypt_mode: CryptMode,
 ) -> Result<c_int, Error> {
     //println!("add config {} size {}", name, size);
 
-    let blob_name = format!("{}.blob", name);
+    if config_blob_name.archive_type() != ArchiveType::Blob {
+        bail!(
+            "cannot add '{}' as config blob.",
+            config_blob_name.to_string()
+        );
+    }
 
     let options = UploadOptions {
         compress,
@@ -109,13 +116,12 @@ pub(crate) async fn add_config(
         ..UploadOptions::default()
     };
 
-    let blob_name: BackupArchiveName = blob_name.parse()?;
     let stats = client
-        .upload_blob_from_data(data, &blob_name, options)
+        .upload_blob_from_data(data, &config_blob_name, options)
         .await?;
 
     let mut guard = manifest.lock().unwrap();
-    guard.add_file(&blob_name, stats.size, stats.csum, crypt_mode)?;
+    guard.add_file(&config_blob_name, stats.size, stats.csum, crypt_mode)?;
 
     Ok(0)
 }
-- 
2.47.3

[PATCH proxmox-backup-qemu v2 5/8] restore: implement `bw-limit` parameter imposing download rate limits

[Christian Ebner]

Add an optional `bw-limit` parameter to allow setting a download
bandwidth limit for pbs-restore. Since this is a breaking api change,
expose this in the public api as new function and refactor to keep
common code deduplicated.

Subsequently this new API function will be extend further by a
additional parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml     |  1 +
 current-api.h  | 13 +++++++++++++
 src/lib.rs     | 49 +++++++++++++++++++++++++++++++++++++++++++++++--
 src/restore.rs |  9 +++++++--
 4 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index d73324f..8b01ee0 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -32,6 +32,7 @@ once_cell = "1.5"
 openssl = "0.10"
 
 proxmox-async = "0.5"
+proxmox-human-byte = "1"
 proxmox-lang = "1"
 proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
diff --git a/current-api.h b/current-api.h
index 60d7558..b9a3a44 100644
--- a/current-api.h
+++ b/current-api.h
@@ -319,6 +319,19 @@ struct ProxmoxRestoreHandle *proxmox_restore_new_ns(const char *repo,
                                                     const char *fingerprint,
                                                     char **error);
 
+/**
+ * Connect to the backup server for restore (sync)
+ */
+struct ProxmoxRestoreHandle *proxmox_restore_new_ns_rate_limited(const char *repo,
+                                                                 const char *snapshot,
+                                                                 const char *namespace_,
+                                                                 const char *password,
+                                                                 const char *keyfile,
+                                                                 const char *key_password,
+                                                                 const char *fingerprint,
+                                                                 const char *bw_limit,
+                                                                 char **error);
+
 /**
  * Open connection to the backup server (sync)
  *
diff --git a/src/lib.rs b/src/lib.rs
index 1d4ea21..4d4c28b 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -3,12 +3,16 @@
 use anyhow::{format_err, Error};
 use std::ffi::CString;
 use std::os::raw::{c_char, c_int, c_long, c_uchar, c_void};
-use std::ptr;
+use std::ptr::{self, null};
 use std::sync::{Arc, Condvar, Mutex};
 
+use proxmox_human_byte::HumanByte;
 use proxmox_lang::try_block;
 
-use pbs_api_types::{Authid, BackupArchiveName, BackupDir, BackupNamespace, BackupType, CryptMode};
+use pbs_api_types::{
+    Authid, BackupArchiveName, BackupDir, BackupNamespace, BackupType, ClientRateLimitConfig,
+    CryptMode,
+};
 use pbs_client::BackupRepository;
 
 pub mod capi_types;
@@ -140,6 +144,7 @@ pub(crate) struct BackupSetup {
     pub key_password: Option<String>,
     pub master_keyfile: Option<std::path::PathBuf>,
     pub fingerprint: Option<String>,
+    pub inbound_rate_limit: Option<ClientRateLimitConfig>,
 }
 
 // helper class to implement synchronous interface
@@ -304,6 +309,7 @@ pub extern "C" fn proxmox_backup_new_ns(
             key_password,
             master_keyfile,
             fingerprint,
+            inbound_rate_limit: None,
         };
 
         BackupTask::new(setup, compress, crypt_mode)
@@ -816,6 +822,7 @@ pub extern "C" fn proxmox_restore_new(
             key_password,
             master_keyfile: None,
             fingerprint,
+            inbound_rate_limit: None,
         };
 
         RestoreTask::new(setup)
@@ -842,6 +849,33 @@ pub extern "C" fn proxmox_restore_new_ns(
     key_password: *const c_char,
     fingerprint: *const c_char,
     error: *mut *mut c_char,
+) -> *mut ProxmoxRestoreHandle {
+    proxmox_restore_new_ns_rate_limited(
+        repo,
+        snapshot,
+        namespace,
+        password,
+        keyfile,
+        key_password,
+        fingerprint,
+        null(),
+        error,
+    )
+}
+
+/// Connect to the backup server for restore (sync)
+#[no_mangle]
+#[allow(clippy::not_unsafe_ptr_arg_deref)]
+pub extern "C" fn proxmox_restore_new_ns_rate_limited(
+    repo: *const c_char,
+    snapshot: *const c_char,
+    namespace: *const c_char,
+    password: *const c_char,
+    keyfile: *const c_char,
+    key_password: *const c_char,
+    fingerprint: *const c_char,
+    bw_limit: *const c_char,
+    error: *mut *mut c_char,
 ) -> *mut ProxmoxRestoreHandle {
     let result: Result<_, Error> = try_block!({
         let repo: BackupRepository = tools::utf8_c_string(repo)?
@@ -862,6 +896,16 @@ pub extern "C" fn proxmox_restore_new_ns(
         let keyfile = tools::utf8_c_string(keyfile)?.map(std::path::PathBuf::from);
         let key_password = tools::utf8_c_string(key_password)?;
         let fingerprint = tools::utf8_c_string(fingerprint)?;
+        let inbound_rate_limit = match tools::utf8_c_string(bw_limit)? {
+            Some(rate_limit) => {
+                let rate_limit: HumanByte = rate_limit.parse()?;
+                Some(ClientRateLimitConfig {
+                    rate: Some(rate_limit),
+                    burst: None,
+                })
+            }
+            None => None,
+        };
 
         let setup = BackupSetup {
             host: repo.host().to_owned(),
@@ -876,6 +920,7 @@ pub extern "C" fn proxmox_restore_new_ns(
             key_password,
             master_keyfile: None,
             fingerprint,
+            inbound_rate_limit,
         };
 
         RestoreTask::new(setup)
diff --git a/src/restore.rs b/src/restore.rs
index 8f518bc..c6c1c9c 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -8,7 +8,7 @@ use tokio::runtime::Runtime;
 
 use proxmox_async::runtime::get_runtime_with_builder;
 
-use pbs_api_types::BackupArchiveName;
+use pbs_api_types::{BackupArchiveName, RateLimitConfig};
 use pbs_client::{BackupReader, HttpClient, HttpClientOptions, RemoteChunkReader};
 use pbs_datastore::cached_chunk_reader::CachedChunkReader;
 use pbs_datastore::data_blob::DataChunkBuilder;
@@ -91,11 +91,16 @@ impl RestoreTask {
     }
 
     pub async fn connect(&self) -> Result<libc::c_int, Error> {
-        let options = HttpClientOptions::new_non_interactive(
+        let mut options = HttpClientOptions::new_non_interactive(
             self.setup.password.clone(),
             self.setup.fingerprint.clone(),
         );
 
+        if let Some(limit) = self.setup.inbound_rate_limit.clone() {
+            let limit = RateLimitConfig::from_client_config(limit);
+            options = options.rate_limit(limit);
+        }
+
         let http = HttpClient::new(
             &self.setup.host,
             self.setup.port,
-- 
2.47.3

[PATCH proxmox-backup-qemu v2 6/8] restore: implement `rate-limit` imposing chunk data stream limits

[Christian Ebner]

In addition to allowing only bandwidth limit downloaded data via the
limits imposed on the http client, also allow to explicitley limit
the chunk stream data when doing restores. By this not only zero
chunks are taken into account for the limit, but also the limit
is applied on the decompressed chunk data.

This further extends the new API function introduced by previous
changes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml     |  1 +
 current-api.h  |  1 +
 src/lib.rs     | 34 ++++++++++++++++++++++++----------
 src/restore.rs | 23 +++++++++++++++++++++++
 4 files changed, 49 insertions(+), 10 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index 8b01ee0..8c3ef26 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -34,6 +34,7 @@ openssl = "0.10"
 proxmox-async = "0.5"
 proxmox-human-byte = "1"
 proxmox-lang = "1"
+proxmox-rate-limiter = "1"
 proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
 proxmox-sys = "1"
diff --git a/current-api.h b/current-api.h
index b9a3a44..25bf981 100644
--- a/current-api.h
+++ b/current-api.h
@@ -330,6 +330,7 @@ struct ProxmoxRestoreHandle *proxmox_restore_new_ns_rate_limited(const char *rep
                                                                  const char *key_password,
                                                                  const char *fingerprint,
                                                                  const char *bw_limit,
+                                                                 const char *rate_limit,
                                                                  char **error);
 
 /**
diff --git a/src/lib.rs b/src/lib.rs
index 4d4c28b..0a37e2f 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -145,6 +145,7 @@ pub(crate) struct BackupSetup {
     pub master_keyfile: Option<std::path::PathBuf>,
     pub fingerprint: Option<String>,
     pub inbound_rate_limit: Option<ClientRateLimitConfig>,
+    pub outbound_rate_limit: Option<ClientRateLimitConfig>,
 }
 
 // helper class to implement synchronous interface
@@ -310,6 +311,7 @@ pub extern "C" fn proxmox_backup_new_ns(
             master_keyfile,
             fingerprint,
             inbound_rate_limit: None,
+            outbound_rate_limit: None,
         };
 
         BackupTask::new(setup, compress, crypt_mode)
@@ -778,6 +780,22 @@ fn restore_handle_to_task(handle: *mut ProxmoxRestoreHandle) -> Arc<RestoreTask>
     Arc::clone(restore_task)
 }
 
+fn client_rate_limit_config_from(
+    limit: *const c_char,
+) -> Result<Option<ClientRateLimitConfig>, Error> {
+    let rate_limit_config = match tools::utf8_c_string(limit)? {
+        Some(rate_limit) => {
+            let rate_limit: HumanByte = rate_limit.parse()?;
+            Some(ClientRateLimitConfig {
+                rate: Some(rate_limit),
+                burst: None,
+            })
+        }
+        None => None,
+    };
+    Ok(rate_limit_config)
+}
+
 /// DEPRECATED: Connect to the backup server for restore (sync)
 ///
 /// Deprecated in favor of `proxmox_restore_new_ns` which includes a namespace parameter.
@@ -823,6 +841,7 @@ pub extern "C" fn proxmox_restore_new(
             master_keyfile: None,
             fingerprint,
             inbound_rate_limit: None,
+            outbound_rate_limit: None,
         };
 
         RestoreTask::new(setup)
@@ -859,6 +878,7 @@ pub extern "C" fn proxmox_restore_new_ns(
         key_password,
         fingerprint,
         null(),
+        null(),
         error,
     )
 }
@@ -875,6 +895,7 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
     key_password: *const c_char,
     fingerprint: *const c_char,
     bw_limit: *const c_char,
+    rate_limit: *const c_char,
     error: *mut *mut c_char,
 ) -> *mut ProxmoxRestoreHandle {
     let result: Result<_, Error> = try_block!({
@@ -896,16 +917,8 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
         let keyfile = tools::utf8_c_string(keyfile)?.map(std::path::PathBuf::from);
         let key_password = tools::utf8_c_string(key_password)?;
         let fingerprint = tools::utf8_c_string(fingerprint)?;
-        let inbound_rate_limit = match tools::utf8_c_string(bw_limit)? {
-            Some(rate_limit) => {
-                let rate_limit: HumanByte = rate_limit.parse()?;
-                Some(ClientRateLimitConfig {
-                    rate: Some(rate_limit),
-                    burst: None,
-                })
-            }
-            None => None,
-        };
+        let inbound_rate_limit = client_rate_limit_config_from(bw_limit)?;
+        let outbound_rate_limit = client_rate_limit_config_from(rate_limit)?;
 
         let setup = BackupSetup {
             host: repo.host().to_owned(),
@@ -921,6 +934,7 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
             master_keyfile: None,
             fingerprint,
             inbound_rate_limit,
+            outbound_rate_limit,
         };
 
         RestoreTask::new(setup)
diff --git a/src/restore.rs b/src/restore.rs
index c6c1c9c..86ec280 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -1,5 +1,6 @@
 use std::convert::TryInto;
 use std::sync::{Arc, Mutex};
+use std::time::Instant;
 
 use anyhow::{bail, format_err, Error};
 use futures::StreamExt;
@@ -7,6 +8,7 @@ use once_cell::sync::OnceCell;
 use tokio::runtime::Runtime;
 
 use proxmox_async::runtime::get_runtime_with_builder;
+use proxmox_rate_limiter::{RateLimit, RateLimiter};
 
 use pbs_api_types::{BackupArchiveName, RateLimitConfig};
 use pbs_client::{BackupReader, HttpClient, HttpClientOptions, RemoteChunkReader};
@@ -43,6 +45,7 @@ pub(crate) struct RestoreTask {
     client: OnceCell<Arc<BackupReader>>,
     manifest: OnceCell<Arc<BackupManifest>>,
     image_registry: Arc<Mutex<Registry<ImageAccessInfo>>>,
+    rate_limiter: Option<Arc<Mutex<RateLimiter>>>,
 }
 
 impl RestoreTask {
@@ -61,6 +64,13 @@ impl RestoreTask {
                 Some(Arc::new(CryptConfig::new(key)?))
             }
         };
+        let rate_limiter = if let Some(limit) = &setup.outbound_rate_limit {
+            limit
+                .rate
+                .map(|rate| Arc::new(Mutex::new(RateLimiter::new(rate.as_u64(), rate.as_u64()))))
+        } else {
+            None
+        };
 
         Ok(Self {
             setup,
@@ -69,6 +79,7 @@ impl RestoreTask {
             client: OnceCell::new(),
             manifest: OnceCell::new(),
             image_registry: Arc::new(Mutex::new(Registry::<ImageAccessInfo>::new())),
+            rate_limiter,
         })
     }
 
@@ -214,6 +225,7 @@ impl RestoreTask {
             let res = res?;
             match res {
                 (None, offset) => {
+                    self.rate_limit_chunk_stream(index.chunk_size as u64).await;
                     let res = write_zero_callback(offset, index.chunk_size as u64);
                     if res < 0 {
                         bail!("write_zero_callback failed ({})", res);
@@ -222,6 +234,7 @@ impl RestoreTask {
                     zeroes += index.chunk_size;
                 }
                 (Some(raw_data), offset) => {
+                    self.rate_limit_chunk_stream(raw_data.len() as u64).await;
                     let res = write_data_callback(offset, &raw_data);
                     if res < 0 {
                         bail!("write_data_callback failed ({})", res);
@@ -260,6 +273,16 @@ impl RestoreTask {
         Ok(())
     }
 
+    async fn rate_limit_chunk_stream(&self, chunk_size: u64) {
+        if let Some(limiter) = &self.rate_limiter {
+            let delay = limiter
+                .lock()
+                .unwrap()
+                .register_traffic(Instant::now(), chunk_size);
+            tokio::time::sleep(delay).await;
+        }
+    }
+
     pub fn get_image_length(&self, aid: u8) -> Result<u64, Error> {
         let mut guard = self.image_registry.lock().unwrap();
         let info = guard.lookup(aid)?;
-- 
2.47.3

[PATCH pve-qemu v2 7/8] fix #7530: pbs-restore: expose optional rate limit parameters

[Christian Ebner]

Implements the flags to impose download and output data stream rate
limits for pbs-restore.

Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=7530
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 ...se-optional-parameters-imposing-band.patch | 78 +++++++++++++++++++
 debian/patches/series                         |  1 +
 2 files changed, 79 insertions(+)
 create mode 100644 debian/patches/pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch

diff --git a/debian/patches/pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch b/debian/patches/pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch
new file mode 100644
index 0000000..3da909b
--- /dev/null
+++ b/debian/patches/pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch
@@ -0,0 +1,78 @@
+From 1e3bcc67823dccbab2d48f874598525502f78f97 Mon Sep 17 00:00:00 2001
+From: Christian Ebner <c.ebner@proxmox.com>
+Date: Mon, 24 Aug 2026 14:40:52 +0200
+Subject: [PATCH] pbs-restore: expose optional parameters imposing bandwidth
+ rate limits
+
+Exposes `bwlimit` and `rate-limit` as newly added optional parameters.
+
+This parameters are now exposes in the proxmox-backup-qemu lib's API
+function and allow to impose bandwidth limits on download by using
+`bwlimit` or output chunk data stream via the `rate-limit`.
+
+The former limits the http client download stream, not taking chunk
+compression and zero chunks into account, while the latter limits the
+output stream, operationg on decompressed raw data including all zero
+chunk data.
+
+Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
+---
+ pbs-restore.c | 14 +++++++++++++-
+ 1 file changed, 13 insertions(+), 1 deletion(-)
+
+diff --git a/pbs-restore.c b/pbs-restore.c
+index 55a3cb235d..4efd18d19e 100644
+--- a/pbs-restore.c
++++ b/pbs-restore.c
+@@ -79,6 +79,8 @@ int main(int argc, char **argv)
+     const char *repository = NULL;
+     const char *backup_ns = NULL;
+     const char *keyfile = NULL;
++    const char *bw_limit = NULL;
++    const char *rate_limit = NULL;
+     int verbose = false;
+     bool skip_zero = false;
+     bool no_cache = false;
+@@ -95,6 +97,8 @@ int main(int argc, char **argv)
+             {"ns", required_argument, 0, 'n'},
+             {"keyfile", required_argument, 0, 'k'},
+             {"no-cache", no_argument, 0, 'N'},
++            {"bwlimit", required_argument, 0, 'l'},
++            {"rate-limit", required_argument, 0, 'L'},
+             {0, 0, 0, 0}
+         };
+         int c = getopt_long(argc, argv, "hvf:r:k:", long_options, NULL);
+@@ -129,6 +133,12 @@ int main(int argc, char **argv)
+             case 'N':
+                 no_cache = true;
+                 break;
++            case 'l':
++                bw_limit = g_strdup(argv[optind - 1]);
++                break;
++            case 'L':
++                rate_limit = g_strdup(argv[optind - 1]);
++                break;
+             case 'h':
+                 help();
+                 return 0;
+@@ -170,7 +180,7 @@ int main(int argc, char **argv)
+         fprintf(stderr, "connecting to repository '%s'\n", repository);
+     }
+     char *pbs_error = NULL;
+-    ProxmoxRestoreHandle *conn = proxmox_restore_new_ns(
++    ProxmoxRestoreHandle *conn = proxmox_restore_new_ns_rate_limited(
+         repository,
+         snapshot,
+         backup_ns,
+@@ -178,6 +188,8 @@ int main(int argc, char **argv)
+         keyfile,
+         key_password,
+         fingerprint,
++        bw_limit,
++        rate_limit,
+         &pbs_error
+     );
+     if (conn == NULL) {
+-- 
+2.47.3
+
diff --git a/debian/patches/series b/debian/patches/series
index 3bc3548..09a93e3 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -61,3 +61,4 @@ pve/0043-PVE-backup-prepare-for-the-switch-to-using-blockdev-.patch
 pve/0044-savevm-async-reuse-migration-blocker-check-for-snaps.patch
 pve/0045-pbs-restore-add-no-cache-flag-to-skip-host-page-cach.patch
 pve/0046-ui-spice-core-work-around-broken-input-cleanup-in-li.patch
+pve/0047-pbs-restore-expose-optional-parameters-imposing-band.patch
-- 
2.47.3

[PATCH qemu-server v2 8/8] pbs-restore: set restore bandwidth limit for volumes

[Christian Ebner]

If configured, pass the optional bandwidth limit parameter to
pbs-restore. Since configured values are provided in multiples
of 1k, extend the value by this scale. By using the chunk stream
output limit imposed by `rate-limit` rather than download bandwidth
limit imposed by `bw-limit`, the restore is limited by output stream
thereby taking decompression and zero chunks into account.

Bumps runtime dependency on pve-qemu in order to assure the flags
are available.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 debian/control        | 2 +-
 src/PVE/QemuServer.pm | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index c236b690..74ccd663 100644
--- a/debian/control
+++ b/debian/control
@@ -61,7 +61,7 @@ Depends: conntrack,
          pve-edk2-firmware-ovmf (>= 4.2025.05-2) [amd64],
          pve-firewall (>= 6.0.3),
          pve-ha-manager (>= 5.0.3),
-         pve-qemu-kvm (>= 11.0.3-3),
+         pve-qemu-kvm (>= 11.0.3-4),
          python3-virt-firmware,
          socat,
          swtpm,
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 759f7db2..2da05f56 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -7036,6 +7036,9 @@ sub restore_proxmox_backup_archive {
 
             push @$pbs_restore_cmd, '--format', $d->{format} if $d->{format};
             push @$pbs_restore_cmd, '--keyfile', $keyfile if -e $keyfile;
+            if (defined($options->{bwlimit})) {
+                push @$pbs_restore_cmd, '--rate-limit', "$options->{bwlimit}KiB";
+            }
 
             if (PVE::Storage::volume_has_feature($storecfg, 'sparseinit', $volid)) {
                 push @$pbs_restore_cmd, '--skip-zero';
-- 
2.47.3