public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup v10 0/6] closes #3071: maintenance mode for datastore
@ 2022-04-12  5:25 Hannes Laimer
  2022-04-12  5:25 ` [pbs-devel] [PATCH proxmox-backup v10 1/6] api-types: add maintenance type Hannes Laimer
                   ` (7 more replies)
  0 siblings, 8 replies; 9+ messages in thread
From: Hannes Laimer @ 2022-04-12  5:25 UTC (permalink / raw)
  To: pbs-devel

Adds maintenance mode and tracking of active reading/writing operations.
The maintenance mode prevents the start of new operations if the type of
operation they would perform on the datastore would conflict with the
maintenance type that is currently set. This check is performed when
lookup_datastore is called. Tasks only call this function once at the
beginning, therefore updating the maintenance type cannot interfere with
already running tasks.

active operations tracking:
Changed file layout to now also keep track of the pid+starttime and the
counts of operations that that pid started, like this it is possible to
not count operations that were started by a dead process, since they 
are also not active anymore. Whenever the file is updated, also entries
of dead processes are removed. When the file is read, only entries of
active processes are counted.

The UI shows a spinner with the count of conflictintg tasks (the tasks
that were started before the maintenance type was updated) next to it.
As soon as all conflicting tasks are finished a checkmark appears.

v10:
 - rebase onto master
 - minor changes suggested by Dylan on v9

v9:
 - bump proxmox-schema dep to 1.2.1
 - loosen MAINTENANCE_MESSAGE_REGEX
 - MaintenanceMode fn check: use and_then()
 - ui: properly (un)escape double quotes in message 

v8:
 - replace enum with MaintenanceMode struct
 - impl check function on MaintenanceMode struct
 - api now encodes type+msg in one string
 - message has to be encoded to ensure proper parsing
(everywhere except the UI the message has to be already encoded
when passed, API+CLI)

v7:
 - tracking: fix counting on clone
 - ui: correct gettext usages + remove usage of capitalize

v6:
 - also use process start time in order to avoid pid clashes(as suggested
 by Thomas and somehow missed by me in the last version)
 - now a single call of get_active_operations return reads and writes
 - improved code structure
 - don't lock when reading

v5:
 - use simple struct and serde instead of manual parsing for file
 - move tracking related stuff into new file (task_tracking.rs)

v4:
 - clones are not also tracked
 - use lockfile, instead of locking the file
 - track pid of the process which started smth
 - updating maintenance mode is now always possible
 - add get_active_operations endpoint for datastore
 - ui: show count of conflicting tasks (or checkmark if no conflicting
     operations are active)

 v3, based on Dominik Csapak <d.csapak@proxmox.com>'s feedback:
 - added Operation enum(r/w), as suggested by
 - added active operation tracking
 - combine type and message into on field

v2:
 - check for maintenance now directly in lookup_datastore
 - parameter for checking is now the last acceptable maintenance type,
   description in commit msg of 2nd patch
 - ui cleanup

Hannes Laimer (6):
  api-types: add maintenance type
  datastore: add check for maintenance in lookup
  pbs-datastore: add active operations tracking
  api: make maintenance_type updatable
  api: add get_active_operations endpoint
  ui: add option to change the maintenance type

 pbs-api-types/Cargo.toml             |   2 +-
 pbs-api-types/src/datastore.rs       |  23 ++++-
 pbs-api-types/src/lib.rs             |   3 +
 pbs-api-types/src/maintenance.rs     |  78 +++++++++++++++++
 pbs-datastore/Cargo.toml             |   1 +
 pbs-datastore/src/datastore.rs       | 126 +++++++++++++++++++--------
 pbs-datastore/src/lib.rs             |   4 +
 pbs-datastore/src/snapshot_reader.rs |   6 +-
 pbs-datastore/src/task_tracking.rs   | 110 +++++++++++++++++++++++
 src/api2/admin/datastore.rs          |  81 +++++++++++------
 src/api2/backup/mod.rs               |   4 +-
 src/api2/config/datastore.rs         |   5 ++
 src/api2/reader/mod.rs               |   6 +-
 src/api2/status.rs                   |   4 +-
 src/api2/tape/backup.rs              |   6 +-
 src/api2/tape/restore.rs             |   6 +-
 src/bin/proxmox-backup-api.rs        |   1 +
 src/bin/proxmox-backup-proxy.rs      |   4 +-
 src/server/mod.rs                    |  16 +++-
 src/server/prune_job.rs              |   4 +-
 src/server/pull.rs                   |   4 +-
 src/server/verify_job.rs             |   4 +-
 www/Makefile                         |   1 +
 www/Utils.js                         |  23 +++++
 www/datastore/OptionView.js          |  30 +++++++
 www/window/MaintenanceOptions.js     |  77 ++++++++++++++++
 26 files changed, 539 insertions(+), 90 deletions(-)
 create mode 100644 pbs-api-types/src/maintenance.rs
 create mode 100644 pbs-datastore/src/task_tracking.rs
 create mode 100644 www/window/MaintenanceOptions.js

-- 
2.30.2





^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-04-12 15:00 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-12  5:25 [pbs-devel] [PATCH proxmox-backup v10 0/6] closes #3071: maintenance mode for datastore Hannes Laimer
2022-04-12  5:25 ` [pbs-devel] [PATCH proxmox-backup v10 1/6] api-types: add maintenance type Hannes Laimer
2022-04-12  5:25 ` [pbs-devel] [PATCH proxmox-backup v10 2/6] datastore: add check for maintenance in lookup Hannes Laimer
2022-04-12  5:25 ` [pbs-devel] [PATCH proxmox-backup v10 3/6] pbs-datastore: add active operations tracking Hannes Laimer
2022-04-12  5:25 ` [pbs-devel] [PATCH proxmox-backup v10 4/6] api: make maintenance_type updatable Hannes Laimer
2022-04-12  5:26 ` [pbs-devel] [PATCH proxmox-backup v10 5/6] api: add get_active_operations endpoint Hannes Laimer
2022-04-12  5:26 ` [pbs-devel] [PATCH proxmox-backup v10 6/6] ui: add option to change the maintenance type Hannes Laimer
2022-04-12  5:40 ` [pbs-devel] [PATCH proxmox-backup v10 0/6] closes #3071: maintenance mode for datastore Dylan Whyte
2022-04-12 15:00 ` [pbs-devel] applied-series: " Thomas Lamprecht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal