[PATCH many 0/8] fix #7530: Implement download bandwidth and restore limits for pbs-restore

[PATCH many 0/8] fix #7530: Implement download bandwidth and restore limits for pbs-restore

[Christian Ebner]

This patch series implements rate limiting functionality for restores
using pbs-restore. In particular it exposes optional `bwlimit` and
`rate-limit` flags, the former to impose download bandwidth limits,
the latter to impose limits on the output data stream, taking into
account zero size chunks and data after decompression.

The output rate limit is further used to enforce the limit when
performing restores via the qemu-server stack, so it is imposed also
when the restore is performed via qmrestore and the PVE UI.

For proxmox-backup-qemu some cleanups and dependencies are bumped as
part of the series as well in order to keep it up-to-date and compile
it cleanly with the latest version available of the proxmox-backup
submodule.

Required build order is:
- pbs-api-types
- proxmox-backup-qemu
- pve-qemu
- qemu-server

Link to the bugtracker issue:
https://bugzilla.proxmox.com/show_bug.cgi?id=7530


proxmox:

Christian Ebner (1):
  pbs-api-types: expose ClientRateLimitConfig fields as pub

 pbs-api-types/src/traffic_control.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


proxmox-backup-qemu:

Christian Ebner (5):
  restore: fix useless borrow clippy warnings for archive names
  commands: fix clippy error for reimplementing Result::ok()
  submodules: bump proxmox-backup submodule to 4.2.5 and fix api changes
  restore: implement `bw-limit` parameter imposing download rate limits
  restore: implement `rate-limit` imposing chunk data stream limits

 Cargo.toml                |  4 ++-
 current-api.h             | 14 +++++++++
 src/backup.rs             |  6 ++--
 src/commands.rs           | 26 +++++-----------
 src/lib.rs                | 63 +++++++++++++++++++++++++++++++++++++--
 src/restore.rs            | 44 ++++++++++++++++++++-------
 submodules/proxmox-backup |  2 +-
 7 files changed, 125 insertions(+), 34 deletions(-)


pve-qemu:

Christian Ebner (1):
  fix #7530: pbs-restore: expose optional rate limit parameters

 ...se-optional-parameters-imposing-band.patch | 78 +++++++++++++++++++
 debian/patches/series                         |  1 +
 2 files changed, 79 insertions(+)
 create mode 100644 debian/patches/pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch


qemu-server:

Christian Ebner (1):
  pbs-restore: set restore bandwidth limit for volumes

 debian/control        | 2 +-
 src/PVE/QemuServer.pm | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)


Summary over all repositories:
  12 files changed, 210 insertions(+), 37 deletions(-)

-- 
Generated by murpp 0.11.0

[PATCH proxmox 1/8] pbs-api-types: expose ClientRateLimitConfig fields as pub

[Christian Ebner]

Required to construct the struct when not deserializing, as will
be used in the PBS client for pbs-restore.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-api-types/src/traffic_control.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pbs-api-types/src/traffic_control.rs b/pbs-api-types/src/traffic_control.rs
index d9ba5e54..34d7b176 100644
--- a/pbs-api-types/src/traffic_control.rs
+++ b/pbs-api-types/src/traffic_control.rs
@@ -96,9 +96,9 @@ const CLIENT_BURST_SCHEMA: Schema = HumanByte::API_SCHEMA
 /// Client Rate Limit Configuration
 pub struct ClientRateLimitConfig {
     #[serde(skip_serializing_if = "Option::is_none")]
-    rate: Option<HumanByte>,
+    pub rate: Option<HumanByte>,
     #[serde(skip_serializing_if = "Option::is_none")]
-    burst: Option<HumanByte>,
+    pub burst: Option<HumanByte>,
 }
 
 #[api(
-- 
2.47.3

[PATCH proxmox-backup-qemu 2/8] restore: fix useless borrow clippy warnings for archive names

[Christian Ebner]

The archive name variable for given callsides already hold a shared
reference only, no need to borrow again.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/restore.rs | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/src/restore.rs b/src/restore.rs
index 6cafd78..bbe0016 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -151,9 +151,7 @@ impl RestoreTask {
             None => bail!("no manifest"),
         };
 
-        let index = client
-            .download_fixed_index(&manifest, &archive_name)
-            .await?;
+        let index = client.download_fixed_index(&manifest, archive_name).await?;
 
         let (_, zero_chunk_digest) = DataChunkBuilder::build_zero_chunk(
             self.crypt_config.as_ref().map(Arc::as_ref),
@@ -163,7 +161,7 @@ impl RestoreTask {
 
         let most_used = index.find_most_used_chunks(8);
 
-        let file_info = manifest.lookup_file_info(&archive_name)?;
+        let file_info = manifest.lookup_file_info(archive_name)?;
 
         let chunk_reader = RemoteChunkReader::new(
             Arc::clone(&client),
@@ -274,13 +272,11 @@ impl RestoreTask {
             None => bail!("no manifest"),
         };
 
-        let index = client
-            .download_fixed_index(&manifest, &archive_name)
-            .await?;
+        let index = client.download_fixed_index(&manifest, archive_name).await?;
         let archive_size = index.index_bytes();
         let most_used = index.find_most_used_chunks(8);
 
-        let file_info = manifest.lookup_file_info(&archive_name)?;
+        let file_info = manifest.lookup_file_info(archive_name)?;
 
         let chunk_reader = RemoteChunkReader::new(
             Arc::clone(&client),
-- 
2.47.3

[PATCH proxmox-backup-qemu 3/8] commands: fix clippy error for reimplementing Result::ok()

[Christian Ebner]

Call Result::ok() on the index download result, avoiding
reimplementation thereof.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/commands.rs | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/src/commands.rs b/src/commands.rs
index 565649f..9b84966 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -195,14 +195,11 @@ pub(crate) async fn register_image(
 
     let index = match manifest {
         Some(manifest) => {
-            match client
+            // not having a previous index is not fatal, so ignore errors
+            client
                 .download_previous_fixed_index(&archive_name, &manifest, Arc::clone(&known_chunks))
                 .await
-            {
-                Ok(index) => Some(index),
-                // not having a previous index is not fatal, so ignore errors
-                Err(_) => None,
-            }
+                .ok()
         }
         None => None,
     };
-- 
2.47.3

[PATCH proxmox-backup-qemu 4/8] submodules: bump proxmox-backup submodule to 4.2.5 and fix api changes

[Christian Ebner]

Bump the submodule to the latest version 4.2.5 and adapt to recent api
changes, which cover in particular:
- The archive name for guest configs which now has the stricter
  BackupArchiveName type as introduced in [0].
- Keeping the signature check of a downloaded previous manifest, made
  optional via a flag in [1].

Due to the stricter checks, add_config() now allows only client side
extensions for blobs and auto-expands the server side `.blob`
extension when required.

While at it, make sure to always use the latest stricter check by
depending on version 1.0.20, which is not strictly required here but
the submodule already depends on it as well and it protects against
building future versions with the still more relaxed archive name
parsing.

The api type changes affect internal API's only, public API's types
remain unchanged.

[0] https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=b57274a5a64791e7245bc0655ce0f133668db145
[1] https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=b80847b554be38f7a9bcae9bf09e14dfb64afa22

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml                |  2 +-
 src/backup.rs             |  6 ++++--
 src/commands.rs           | 17 +++++------------
 submodules/proxmox-backup |  2 +-
 4 files changed, 11 insertions(+), 16 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index 2719a80..81c46b3 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -37,7 +37,7 @@ proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
 proxmox-sys = "1"
 
-pbs-api-types  = { version = "1" }
+pbs-api-types  = { version = "1.0.20" }
 
 pbs-client     = { path = "submodules/proxmox-backup/pbs-client" }
 pbs-datastore  = { path = "submodules/proxmox-backup/pbs-datastore" }
diff --git a/src/backup.rs b/src/backup.rs
index cf1806a..9f1848b 100644
--- a/src/backup.rs
+++ b/src/backup.rs
@@ -2,6 +2,7 @@ use anyhow::{bail, format_err, Error};
 use once_cell::sync::OnceCell;
 use std::collections::HashSet;
 use std::os::raw::c_int;
+use std::str::FromStr;
 use std::sync::{Arc, Mutex};
 
 use futures::future::{Either, Future, FutureExt};
@@ -10,7 +11,7 @@ use tokio::runtime::Runtime;
 use proxmox_async::runtime::get_runtime_with_builder;
 use proxmox_sys::fs::file_get_contents;
 
-use pbs_api_types::{BackupType, CryptMode};
+use pbs_api_types::{BackupArchiveName, BackupType, CryptMode};
 use pbs_client::{BackupWriter, BackupWriterOptions, HttpClient, HttpClientOptions};
 use pbs_datastore::BackupManifest;
 use pbs_key_config::{load_and_decrypt_key, rsa_encrypt_key_config, KeyConfig};
@@ -157,7 +158,7 @@ impl BackupTask {
             )
             .await?;
 
-            let last_manifest = writer.download_previous_manifest().await;
+            let last_manifest = writer.download_previous_manifest(true).await;
             let mut result = 0;
             if let Ok(last_manifest) = last_manifest {
                 result = 1;
@@ -186,6 +187,7 @@ impl BackupTask {
 
     pub async fn add_config(&self, name: String, data: Vec<u8>) -> Result<c_int, Error> {
         self.check_aborted()?;
+        let name = BackupArchiveName::from_str(&name)?;
 
         let command_future = add_config(
             self.need_writer()?,
diff --git a/src/commands.rs b/src/commands.rs
index 9b84966..3c2726c 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -94,15 +94,13 @@ async fn register_zero_chunk(
 pub(crate) async fn add_config(
     client: Arc<BackupWriter>,
     manifest: Arc<Mutex<BackupManifest>>,
-    name: String,
+    config_blob_name: BackupArchiveName,
     data: Vec<u8>,
     compress: bool,
     crypt_mode: CryptMode,
 ) -> Result<c_int, Error> {
     //println!("add config {} size {}", name, size);
 
-    let blob_name = format!("{}.blob", name);
-
     let options = UploadOptions {
         compress,
         encrypt: crypt_mode == CryptMode::Encrypt,
@@ -110,12 +108,11 @@ pub(crate) async fn add_config(
     };
 
     let stats = client
-        .upload_blob_from_data(data, &blob_name, options)
+        .upload_blob_from_data(data, &config_blob_name, options)
         .await?;
-    let blob_name: BackupArchiveName = blob_name.parse()?;
 
     let mut guard = manifest.lock().unwrap();
-    guard.add_file(&blob_name, stats.size, stats.csum, crypt_mode)?;
+    guard.add_file(&config_blob_name, stats.size, stats.csum, crypt_mode)?;
 
     Ok(0)
 }
@@ -481,7 +478,7 @@ pub(crate) async fn finish_backup(
             ..UploadOptions::default()
         };
         let stats = client
-            .upload_blob_from_data(rsa_encrypted_key, &target.to_string(), options)
+            .upload_blob_from_data(rsa_encrypted_key, target, options)
             .await?;
         manifest
             .lock()
@@ -515,11 +512,7 @@ pub(crate) async fn finish_backup(
     };
 
     client
-        .upload_blob_from_data(
-            manifest.into_bytes(),
-            &MANIFEST_BLOB_NAME.to_string(),
-            options,
-        )
+        .upload_blob_from_data(manifest.into_bytes(), &MANIFEST_BLOB_NAME, options)
         .await?;
 
     client.finish().await?;
diff --git a/submodules/proxmox-backup b/submodules/proxmox-backup
index b48af88..5f890fc 160000
--- a/submodules/proxmox-backup
+++ b/submodules/proxmox-backup
@@ -1 +1 @@
-Subproject commit b48af880bee8c654dff29ec2d64b977e224896bb
+Subproject commit 5f890fcb6dc07148c22568ad03bf9c079bcad249
-- 
2.47.3

[PATCH proxmox-backup-qemu 5/8] restore: implement `bw-limit` parameter imposing download rate limits

[Christian Ebner]

Add an optional `bw-limit` parameter to allow setting a download
bandwidth limit for pbs-restore. Since this is a breaking api change,
expose this in the public api as new function and refactor to keep
common code deduplicated.

Subsequently this new API function will be extend further by a
additional parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml     |  1 +
 current-api.h  | 13 +++++++++++++
 src/lib.rs     | 49 +++++++++++++++++++++++++++++++++++++++++++++++--
 src/restore.rs |  9 +++++++--
 4 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index 81c46b3..af7a84a 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -32,6 +32,7 @@ once_cell = "1.5"
 openssl = "0.10"
 
 proxmox-async = "0.5"
+proxmox-human-byte = "1"
 proxmox-lang = "1"
 proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
diff --git a/current-api.h b/current-api.h
index 60d7558..b9a3a44 100644
--- a/current-api.h
+++ b/current-api.h
@@ -319,6 +319,19 @@ struct ProxmoxRestoreHandle *proxmox_restore_new_ns(const char *repo,
                                                     const char *fingerprint,
                                                     char **error);
 
+/**
+ * Connect to the backup server for restore (sync)
+ */
+struct ProxmoxRestoreHandle *proxmox_restore_new_ns_rate_limited(const char *repo,
+                                                                 const char *snapshot,
+                                                                 const char *namespace_,
+                                                                 const char *password,
+                                                                 const char *keyfile,
+                                                                 const char *key_password,
+                                                                 const char *fingerprint,
+                                                                 const char *bw_limit,
+                                                                 char **error);
+
 /**
  * Open connection to the backup server (sync)
  *
diff --git a/src/lib.rs b/src/lib.rs
index 1d4ea21..4d4c28b 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -3,12 +3,16 @@
 use anyhow::{format_err, Error};
 use std::ffi::CString;
 use std::os::raw::{c_char, c_int, c_long, c_uchar, c_void};
-use std::ptr;
+use std::ptr::{self, null};
 use std::sync::{Arc, Condvar, Mutex};
 
+use proxmox_human_byte::HumanByte;
 use proxmox_lang::try_block;
 
-use pbs_api_types::{Authid, BackupArchiveName, BackupDir, BackupNamespace, BackupType, CryptMode};
+use pbs_api_types::{
+    Authid, BackupArchiveName, BackupDir, BackupNamespace, BackupType, ClientRateLimitConfig,
+    CryptMode,
+};
 use pbs_client::BackupRepository;
 
 pub mod capi_types;
@@ -140,6 +144,7 @@ pub(crate) struct BackupSetup {
     pub key_password: Option<String>,
     pub master_keyfile: Option<std::path::PathBuf>,
     pub fingerprint: Option<String>,
+    pub inbound_rate_limit: Option<ClientRateLimitConfig>,
 }
 
 // helper class to implement synchronous interface
@@ -304,6 +309,7 @@ pub extern "C" fn proxmox_backup_new_ns(
             key_password,
             master_keyfile,
             fingerprint,
+            inbound_rate_limit: None,
         };
 
         BackupTask::new(setup, compress, crypt_mode)
@@ -816,6 +822,7 @@ pub extern "C" fn proxmox_restore_new(
             key_password,
             master_keyfile: None,
             fingerprint,
+            inbound_rate_limit: None,
         };
 
         RestoreTask::new(setup)
@@ -842,6 +849,33 @@ pub extern "C" fn proxmox_restore_new_ns(
     key_password: *const c_char,
     fingerprint: *const c_char,
     error: *mut *mut c_char,
+) -> *mut ProxmoxRestoreHandle {
+    proxmox_restore_new_ns_rate_limited(
+        repo,
+        snapshot,
+        namespace,
+        password,
+        keyfile,
+        key_password,
+        fingerprint,
+        null(),
+        error,
+    )
+}
+
+/// Connect to the backup server for restore (sync)
+#[no_mangle]
+#[allow(clippy::not_unsafe_ptr_arg_deref)]
+pub extern "C" fn proxmox_restore_new_ns_rate_limited(
+    repo: *const c_char,
+    snapshot: *const c_char,
+    namespace: *const c_char,
+    password: *const c_char,
+    keyfile: *const c_char,
+    key_password: *const c_char,
+    fingerprint: *const c_char,
+    bw_limit: *const c_char,
+    error: *mut *mut c_char,
 ) -> *mut ProxmoxRestoreHandle {
     let result: Result<_, Error> = try_block!({
         let repo: BackupRepository = tools::utf8_c_string(repo)?
@@ -862,6 +896,16 @@ pub extern "C" fn proxmox_restore_new_ns(
         let keyfile = tools::utf8_c_string(keyfile)?.map(std::path::PathBuf::from);
         let key_password = tools::utf8_c_string(key_password)?;
         let fingerprint = tools::utf8_c_string(fingerprint)?;
+        let inbound_rate_limit = match tools::utf8_c_string(bw_limit)? {
+            Some(rate_limit) => {
+                let rate_limit: HumanByte = rate_limit.parse()?;
+                Some(ClientRateLimitConfig {
+                    rate: Some(rate_limit),
+                    burst: None,
+                })
+            }
+            None => None,
+        };
 
         let setup = BackupSetup {
             host: repo.host().to_owned(),
@@ -876,6 +920,7 @@ pub extern "C" fn proxmox_restore_new_ns(
             key_password,
             master_keyfile: None,
             fingerprint,
+            inbound_rate_limit,
         };
 
         RestoreTask::new(setup)
diff --git a/src/restore.rs b/src/restore.rs
index bbe0016..fff0465 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -8,7 +8,7 @@ use tokio::runtime::Runtime;
 
 use proxmox_async::runtime::get_runtime_with_builder;
 
-use pbs_api_types::BackupArchiveName;
+use pbs_api_types::{BackupArchiveName, RateLimitConfig};
 use pbs_client::{BackupReader, HttpClient, HttpClientOptions, RemoteChunkReader};
 use pbs_datastore::cached_chunk_reader::CachedChunkReader;
 use pbs_datastore::data_blob::DataChunkBuilder;
@@ -91,11 +91,16 @@ impl RestoreTask {
     }
 
     pub async fn connect(&self) -> Result<libc::c_int, Error> {
-        let options = HttpClientOptions::new_non_interactive(
+        let mut options = HttpClientOptions::new_non_interactive(
             self.setup.password.clone(),
             self.setup.fingerprint.clone(),
         );
 
+        if let Some(limit) = self.setup.inbound_rate_limit.clone() {
+            let limit = RateLimitConfig::from_client_config(limit);
+            options = options.rate_limit(limit);
+        }
+
         let http = HttpClient::new(
             &self.setup.host,
             self.setup.port,
-- 
2.47.3

[PATCH proxmox-backup-qemu 6/8] restore: implement `rate-limit` imposing chunk data stream limits

[Christian Ebner]

In addition to allowing only bandwidth limit downloaded data via the
limits imposed on the http client, also allow to explicitley limit
the chunk stream data when doing restores. By this not only zero
chunks are taken into account for the limit, but also the limit
is applied on the decompressed chunk data.

This further extends the new API function introduced by previous
changes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 Cargo.toml     |  1 +
 current-api.h  |  1 +
 src/lib.rs     | 34 ++++++++++++++++++++++++----------
 src/restore.rs | 23 +++++++++++++++++++++++
 4 files changed, 49 insertions(+), 10 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index af7a84a..1b0af2e 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -34,6 +34,7 @@ openssl = "0.10"
 proxmox-async = "0.5"
 proxmox-human-byte = "1"
 proxmox-lang = "1"
+proxmox-rate-limiter = "1"
 proxmox-schema = { version = "5", features = [ "api-macro" ] }
 proxmox-sortable-macro = "1"
 proxmox-sys = "1"
diff --git a/current-api.h b/current-api.h
index b9a3a44..25bf981 100644
--- a/current-api.h
+++ b/current-api.h
@@ -330,6 +330,7 @@ struct ProxmoxRestoreHandle *proxmox_restore_new_ns_rate_limited(const char *rep
                                                                  const char *key_password,
                                                                  const char *fingerprint,
                                                                  const char *bw_limit,
+                                                                 const char *rate_limit,
                                                                  char **error);
 
 /**
diff --git a/src/lib.rs b/src/lib.rs
index 4d4c28b..0a37e2f 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -145,6 +145,7 @@ pub(crate) struct BackupSetup {
     pub master_keyfile: Option<std::path::PathBuf>,
     pub fingerprint: Option<String>,
     pub inbound_rate_limit: Option<ClientRateLimitConfig>,
+    pub outbound_rate_limit: Option<ClientRateLimitConfig>,
 }
 
 // helper class to implement synchronous interface
@@ -310,6 +311,7 @@ pub extern "C" fn proxmox_backup_new_ns(
             master_keyfile,
             fingerprint,
             inbound_rate_limit: None,
+            outbound_rate_limit: None,
         };
 
         BackupTask::new(setup, compress, crypt_mode)
@@ -778,6 +780,22 @@ fn restore_handle_to_task(handle: *mut ProxmoxRestoreHandle) -> Arc<RestoreTask>
     Arc::clone(restore_task)
 }
 
+fn client_rate_limit_config_from(
+    limit: *const c_char,
+) -> Result<Option<ClientRateLimitConfig>, Error> {
+    let rate_limit_config = match tools::utf8_c_string(limit)? {
+        Some(rate_limit) => {
+            let rate_limit: HumanByte = rate_limit.parse()?;
+            Some(ClientRateLimitConfig {
+                rate: Some(rate_limit),
+                burst: None,
+            })
+        }
+        None => None,
+    };
+    Ok(rate_limit_config)
+}
+
 /// DEPRECATED: Connect to the backup server for restore (sync)
 ///
 /// Deprecated in favor of `proxmox_restore_new_ns` which includes a namespace parameter.
@@ -823,6 +841,7 @@ pub extern "C" fn proxmox_restore_new(
             master_keyfile: None,
             fingerprint,
             inbound_rate_limit: None,
+            outbound_rate_limit: None,
         };
 
         RestoreTask::new(setup)
@@ -859,6 +878,7 @@ pub extern "C" fn proxmox_restore_new_ns(
         key_password,
         fingerprint,
         null(),
+        null(),
         error,
     )
 }
@@ -875,6 +895,7 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
     key_password: *const c_char,
     fingerprint: *const c_char,
     bw_limit: *const c_char,
+    rate_limit: *const c_char,
     error: *mut *mut c_char,
 ) -> *mut ProxmoxRestoreHandle {
     let result: Result<_, Error> = try_block!({
@@ -896,16 +917,8 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
         let keyfile = tools::utf8_c_string(keyfile)?.map(std::path::PathBuf::from);
         let key_password = tools::utf8_c_string(key_password)?;
         let fingerprint = tools::utf8_c_string(fingerprint)?;
-        let inbound_rate_limit = match tools::utf8_c_string(bw_limit)? {
-            Some(rate_limit) => {
-                let rate_limit: HumanByte = rate_limit.parse()?;
-                Some(ClientRateLimitConfig {
-                    rate: Some(rate_limit),
-                    burst: None,
-                })
-            }
-            None => None,
-        };
+        let inbound_rate_limit = client_rate_limit_config_from(bw_limit)?;
+        let outbound_rate_limit = client_rate_limit_config_from(rate_limit)?;
 
         let setup = BackupSetup {
             host: repo.host().to_owned(),
@@ -921,6 +934,7 @@ pub extern "C" fn proxmox_restore_new_ns_rate_limited(
             master_keyfile: None,
             fingerprint,
             inbound_rate_limit,
+            outbound_rate_limit,
         };
 
         RestoreTask::new(setup)
diff --git a/src/restore.rs b/src/restore.rs
index fff0465..6c1a0c6 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -1,5 +1,6 @@
 use std::convert::TryInto;
 use std::sync::{Arc, Mutex};
+use std::time::Instant;
 
 use anyhow::{bail, format_err, Error};
 use futures::StreamExt;
@@ -7,6 +8,7 @@ use once_cell::sync::OnceCell;
 use tokio::runtime::Runtime;
 
 use proxmox_async::runtime::get_runtime_with_builder;
+use proxmox_rate_limiter::{RateLimit, RateLimiter};
 
 use pbs_api_types::{BackupArchiveName, RateLimitConfig};
 use pbs_client::{BackupReader, HttpClient, HttpClientOptions, RemoteChunkReader};
@@ -43,6 +45,7 @@ pub(crate) struct RestoreTask {
     client: OnceCell<Arc<BackupReader>>,
     manifest: OnceCell<Arc<BackupManifest>>,
     image_registry: Arc<Mutex<Registry<ImageAccessInfo>>>,
+    rate_limiter: Option<Arc<Mutex<RateLimiter>>>,
 }
 
 impl RestoreTask {
@@ -61,6 +64,13 @@ impl RestoreTask {
                 Some(Arc::new(CryptConfig::new(key)?))
             }
         };
+        let rate_limiter = if let Some(limit) = &setup.outbound_rate_limit {
+            limit
+                .rate
+                .map(|rate| Arc::new(Mutex::new(RateLimiter::new(rate.as_u64(), rate.as_u64()))))
+        } else {
+            None
+        };
 
         Ok(Self {
             setup,
@@ -69,6 +79,7 @@ impl RestoreTask {
             client: OnceCell::new(),
             manifest: OnceCell::new(),
             image_registry: Arc::new(Mutex::new(Registry::<ImageAccessInfo>::new())),
+            rate_limiter,
         })
     }
 
@@ -214,6 +225,7 @@ impl RestoreTask {
             let res = res?;
             match res {
                 (None, offset) => {
+                    self.rate_limit_chunk_stream(index.chunk_size as u64).await;
                     let res = write_zero_callback(offset, index.chunk_size as u64);
                     if res < 0 {
                         bail!("write_zero_callback failed ({})", res);
@@ -222,6 +234,7 @@ impl RestoreTask {
                     zeroes += index.chunk_size;
                 }
                 (Some(raw_data), offset) => {
+                    self.rate_limit_chunk_stream(raw_data.len() as u64).await;
                     let res = write_data_callback(offset, &raw_data);
                     if res < 0 {
                         bail!("write_data_callback failed ({})", res);
@@ -260,6 +273,16 @@ impl RestoreTask {
         Ok(())
     }
 
+    async fn rate_limit_chunk_stream(&self, chunk_size: u64) {
+        if let Some(limiter) = &self.rate_limiter {
+            let delay = limiter
+                .lock()
+                .unwrap()
+                .register_traffic(Instant::now(), chunk_size);
+            tokio::time::sleep(delay).await;
+        }
+    }
+
     pub fn get_image_length(&self, aid: u8) -> Result<u64, Error> {
         let mut guard = self.image_registry.lock().unwrap();
         let info = guard.lookup(aid)?;
-- 
2.47.3

[PATCH pve-qemu 7/8] fix #7530: pbs-restore: expose optional rate limit parameters

[Christian Ebner]

Implements the flags to impose download and output data stream rate
limits for pbs-restore.

Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=7530
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 ...se-optional-parameters-imposing-band.patch | 78 +++++++++++++++++++
 debian/patches/series                         |  1 +
 2 files changed, 79 insertions(+)
 create mode 100644 debian/patches/pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch

diff --git a/debian/patches/pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch b/debian/patches/pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch
new file mode 100644
index 0000000..3da909b
--- /dev/null
+++ b/debian/patches/pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch
@@ -0,0 +1,78 @@
+From 1e3bcc67823dccbab2d48f874598525502f78f97 Mon Sep 17 00:00:00 2001
+From: Christian Ebner <c.ebner@proxmox.com>
+Date: Mon, 24 Aug 2026 14:40:52 +0200
+Subject: [PATCH] pbs-restore: expose optional parameters imposing bandwidth
+ rate limits
+
+Exposes `bwlimit` and `rate-limit` as newly added optional parameters.
+
+This parameters are now exposes in the proxmox-backup-qemu lib's API
+function and allow to impose bandwidth limits on download by using
+`bwlimit` or output chunk data stream via the `rate-limit`.
+
+The former limits the http client download stream, not taking chunk
+compression and zero chunks into account, while the latter limits the
+output stream, operationg on decompressed raw data including all zero
+chunk data.
+
+Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
+---
+ pbs-restore.c | 14 +++++++++++++-
+ 1 file changed, 13 insertions(+), 1 deletion(-)
+
+diff --git a/pbs-restore.c b/pbs-restore.c
+index 55a3cb235d..4efd18d19e 100644
+--- a/pbs-restore.c
++++ b/pbs-restore.c
+@@ -79,6 +79,8 @@ int main(int argc, char **argv)
+     const char *repository = NULL;
+     const char *backup_ns = NULL;
+     const char *keyfile = NULL;
++    const char *bw_limit = NULL;
++    const char *rate_limit = NULL;
+     int verbose = false;
+     bool skip_zero = false;
+     bool no_cache = false;
+@@ -95,6 +97,8 @@ int main(int argc, char **argv)
+             {"ns", required_argument, 0, 'n'},
+             {"keyfile", required_argument, 0, 'k'},
+             {"no-cache", no_argument, 0, 'N'},
++            {"bwlimit", required_argument, 0, 'l'},
++            {"rate-limit", required_argument, 0, 'L'},
+             {0, 0, 0, 0}
+         };
+         int c = getopt_long(argc, argv, "hvf:r:k:", long_options, NULL);
+@@ -129,6 +133,12 @@ int main(int argc, char **argv)
+             case 'N':
+                 no_cache = true;
+                 break;
++            case 'l':
++                bw_limit = g_strdup(argv[optind - 1]);
++                break;
++            case 'L':
++                rate_limit = g_strdup(argv[optind - 1]);
++                break;
+             case 'h':
+                 help();
+                 return 0;
+@@ -170,7 +180,7 @@ int main(int argc, char **argv)
+         fprintf(stderr, "connecting to repository '%s'\n", repository);
+     }
+     char *pbs_error = NULL;
+-    ProxmoxRestoreHandle *conn = proxmox_restore_new_ns(
++    ProxmoxRestoreHandle *conn = proxmox_restore_new_ns_rate_limited(
+         repository,
+         snapshot,
+         backup_ns,
+@@ -178,6 +188,8 @@ int main(int argc, char **argv)
+         keyfile,
+         key_password,
+         fingerprint,
++        bw_limit,
++        rate_limit,
+         &pbs_error
+     );
+     if (conn == NULL) {
+-- 
+2.47.3
+
diff --git a/debian/patches/series b/debian/patches/series
index 6ef37f4..947b48d 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -77,3 +77,4 @@ pve/0044-PVE-backup-implement-backup-access-setup-and-teardow.patch
 pve/0045-PVE-backup-prepare-for-the-switch-to-using-blockdev-.patch
 pve/0046-savevm-async-reuse-migration-blocker-check-for-snaps.patch
 pve/0047-pbs-restore-add-no-cache-flag-to-skip-host-page-cach.patch
+pve/0048-pbs-restore-expose-optional-parameters-imposing-band.patch
-- 
2.47.3

[PATCH qemu-server 8/8] pbs-restore: set restore bandwidth limit for volumes

[Christian Ebner]

If configured, pass the optional bandwidth limit parameter to
pbs-restore. Since configured values are provided in multiples
of 1k, extend the value by this scale. By using the chunk stream
output limit imposed by `rate-limit` rather than download bandwidth
limit imposed by `bw-limit`, the restore is limited by output stream
thereby taking decompression and zero chunks into account.

Bumps runtime dependency on pve-qemu in order to assure the flags
are available.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 debian/control        | 2 +-
 src/PVE/QemuServer.pm | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index a9949d5a..bd7c4b37 100644
--- a/debian/control
+++ b/debian/control
@@ -61,7 +61,7 @@ Depends: conntrack,
          pve-edk2-firmware-ovmf (>= 4.2025.05-2) [amd64],
          pve-firewall (>= 6.0.3),
          pve-ha-manager (>= 5.0.3),
-         pve-qemu-kvm (>= 11.0.3-3),
+         pve-qemu-kvm (>= 11.0.3-4),
          python3-virt-firmware,
          socat,
          swtpm,
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 63d8c135..e873d602 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -7086,6 +7086,9 @@ sub restore_proxmox_backup_archive {
 
             push @$pbs_restore_cmd, '--format', $d->{format} if $d->{format};
             push @$pbs_restore_cmd, '--keyfile', $keyfile if -e $keyfile;
+            if (defined($options->{bwlimit})) {
+                push @$pbs_restore_cmd, '--rate-limit', "$options->{bwlimit}KiB";
+            }
 
             if (PVE::Storage::volume_has_feature($storecfg, 'sparseinit', $volid)) {
                 push @$pbs_restore_cmd, '--skip-zero';
-- 
2.47.3