[pbs-devel] [PATCH proxmox/proxmox-backup] Rate Limiter Implementation

[Dietmar Maurer <dietmar@proxmox.com>]

This implements a simple Token Bucket based rate limiter. That filter
can be used to:

- limit http client connection
- limit speed at server side (proxmox-backup-proxy)

There is also a new configuration file to configure the server side
rate limits: /etc/proxmox-backup/traffic-control.cfg

The server side filter dynamically updates when the user changes
configuration (even for existing connections).


Dietmar Maurer (proxmox/7):
  Implement a rate limiting stream (AsyncRead, AsyncWrite)
  RateLimitedStream: implement poll_write_vectored
  HttpsConnector: use RateLimitedStream
  RateLimitedStream: allow periodic limiter updates
  RateLimiter: avoid panic in time computations
  RateLimitedStream: implement peer_addr
  RateLimiter: add update_rate method

 proxmox-http/src/client/connector.rs          |  51 +++-
 proxmox-http/src/client/mod.rs                |   6 +
 .../src/client/rate_limited_stream.rs         | 233 ++++++++++++++++++
 proxmox-http/src/client/rate_limiter.rs       |  84 +++++++
 4 files changed, 366 insertions(+), 8 deletions(-)
 create mode 100644 proxmox-http/src/client/rate_limited_stream.rs
 create mode 100644 proxmox-http/src/client/rate_limiter.rs

Dietmar Maurer (proxmox-backup/9):
  pbs-client: add option to use the new RateLimiter
  proxmox-backup-client: add rate/burst parameter to backup CLI
  implement Servive for RateLimitedStream
  New DailyDuration type with nom parser
  DailyDuration: implement time_match()
  Add traffic control configuration config with API
  traffic_control: use Memcom to track. config versions
  implement a traffic control cache for fast rate control limiter
    lockups
  proxmox-backup-proxy: implement traffic control

 Cargo.toml                                    |   1 +
 pbs-api-types/src/lib.rs                      |   7 +
 pbs-api-types/src/traffic_control.rs          |  81 +++++
 pbs-client/src/http_client.rs                 |  24 +-
 pbs-client/src/tools/mod.rs                   |  23 +-
 pbs-config/src/lib.rs                         |   3 +-
 pbs-config/src/memcom.rs                      |  14 +
 pbs-config/src/traffic_control.rs             |  98 ++++++
 proxmox-backup-client/src/main.rs             |  19 +-
 proxmox-rest-server/Cargo.toml                |   1 +
 proxmox-rest-server/src/rest.rs               |  28 ++
 proxmox-systemd/src/daily_duration.rs         | 152 ++++++++++
 proxmox-systemd/src/lib.rs                    |   1 +
 proxmox-systemd/src/parse_time.rs             |  56 ++++
 proxmox-systemd/src/time.rs                   |   2 +-
 src/api2/config/mod.rs                        |   2 +
 src/api2/config/traffic_control.rs            | 283 ++++++++++++++++++
 src/bin/proxmox-backup-manager.rs             |   1 +
 src/bin/proxmox-backup-proxy.rs               |  25 +-
 src/bin/proxmox_backup_manager/mod.rs         |   2 +
 .../proxmox_backup_manager/traffic_control.rs | 105 +++++++
 src/cached_traffic_control.rs                 | 240 +++++++++++++++
 src/lib.rs                                    |   3 +
 23 files changed, 1161 insertions(+), 10 deletions(-)
 create mode 100644 pbs-api-types/src/traffic_control.rs
 create mode 100644 pbs-config/src/traffic_control.rs
 create mode 100644 proxmox-systemd/src/daily_duration.rs
 create mode 100644 src/api2/config/traffic_control.rs
 create mode 100644 src/bin/proxmox_backup_manager/traffic_control.rs
 create mode 100644 src/cached_traffic_control.rs

-- 
2.30.2