[pbs-devel] [PATCH proxmox{, -backup} 0/6] shared rate limiter for s3 client instances

[Christian Ebner <c.ebner@proxmox.com>]

This patches implement a shared bandwidth rate limiter for the s3
client instances, with the goal to allow users to avoid network
congestion by traffic to the s3 backend.

The limiter is shared accorss all the clients using the same s3
endpoint id, setting the limits on client instantiation. To utilize
the pre-existing shared rate limiter implementation from PBS, factor
it out into proxmox-http as this already contains the non-shared rate
limiter implementation.

Expose the rate limit configuration in the s3 endpoint configuration
and provide it to the client's https connector. Expose the settings
also in the advanced options of the s3 endpoint edit/crate window.

proxmox:

Christian Ebner (2):
  http: factor out PBS shared rate limiter implementation
  s3-client: add shared rate limiter via https connector

 proxmox-http/Cargo.toml                 |   7 ++
 proxmox-http/debian/control             |  18 ++++
 proxmox-http/src/lib.rs                 |   5 +
 proxmox-http/src/shared_rate_limiter.rs | 129 ++++++++++++++++++++++++
 proxmox-s3-client/Cargo.toml            |   5 +-
 proxmox-s3-client/debian/control        |   4 +
 proxmox-s3-client/examples/s3_client.rs |   1 +
 proxmox-s3-client/src/api_types.rs      |  29 ++++++
 proxmox-s3-client/src/client.rs         |  62 +++++++++++-
 proxmox-s3-client/src/lib.rs            |   4 +-
 10 files changed, 259 insertions(+), 5 deletions(-)
 create mode 100644 proxmox-http/src/shared_rate_limiter.rs


proxmox-backup:

Christian Ebner (4):
  traffic control: use factored out shared rate limiter
  api: config: update s3 endpoint rate limits in config
  datastore: s3: set rate limiter options for s3 client
  ui: expose rate and burst limits for s3 endpoints

 Cargo.toml                       |   2 +-
 pbs-datastore/src/datastore.rs   |  18 ++++-
 src/api2/admin/s3.rs             |   9 ++-
 src/api2/config/s3.rs            |  34 ++++++++-
 src/tools/mod.rs                 |   4 -
 src/tools/shared_rate_limiter.rs | 122 -------------------------------
 src/traffic_control_cache.rs     |   8 +-
 www/window/S3ClientEdit.js       |  34 +++++++++
 8 files changed, 96 insertions(+), 135 deletions(-)
 delete mode 100644 src/tools/shared_rate_limiter.rs


Summary over all repositories:
  18 files changed, 355 insertions(+), 140 deletions(-)

-- 
Generated by git-murpp 0.8.1


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel