From: Dominik Csapak <d.csapak@proxmox.com>
To: Proxmox Backup Server development discussion
<pbs-devel@lists.proxmox.com>,
Hannes Laimer <h.laimer@proxmox.com>
Subject: Re: [pbs-devel] [PATCH proxmox-backup v6 0/6] closes #3071: maintenance mode for datastore
Date: Thu, 3 Feb 2022 11:53:30 +0100 [thread overview]
Message-ID: <b6ec05aa-18f3-5da5-befb-753150444a00@proxmox.com> (raw)
In-Reply-To: <20220202154927.30572-1-h.laimer@proxmox.com>
hi,
it seems the tape patches are squashed into the wrong patch 3/6 instead
of 2/6, but no big issue imho
i found some issues that were already there last time around (sorry)
but i think those are small enough that you can send a follow up..
aside from the issues in 3/6 and 6/6 consider it
(note, i tested it with my suggestion in 3/6)
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
On 2/2/22 16:49, Hannes Laimer wrote:
> Adds maintenance mode and tracking of active reading/writing operations.
> The maintenance mode prevents the start of new operations if the type of
> operation they would perform on the datastore would conflict with the
> maintenance type that is currently set. This check is performed when
> lookup_datastore is called. Tasks only call this function once at the
> beginning, therefore updating the maintenance type cannot interfere with
> already running tasks.
>
> active operations tracking:
> Changed file layout to now also keep track of the pid+starttime and the
> counts of operations that that pid started, like this it is possible to
> not count operations that were started by a dead process, since they
> are also not active anymore. Whenever the file is updated, also entries
> of dead processes are removed. When the file is read, only entries of
> active processes are counted.
>
> The UI shows a spinner with the count of conflictintg tasks (the tasks
> that were started before the maintenance type was updated) next to it.
> As soon as all conflicting tasks are finished a checkmark appears.
>
> v6:
> - also use process start time in order to avoid pid clashes(as suggested
> by Thomas and somehow missed by me in the last version)
> - now a single call of get_active_operations return reads and writes
> - improved code structure
> - don't lock when reading
>
> v5:
> - use simple struct and serde instead of manual parsing for file
> - move tracking related stuff into new file (task_tracking.rs)
>
> v4:
> - clones are not also tracked
> - use lockfile, instead of locking the file
> - track pid of the process which started smth
> - updating maintenance mode is now always possible
> - add get_active_operations endpoint for datastore
> - ui: show count of conflicting tasks (or checkmark if no conflicting
> operations are active)
>
> v3, based on Dominik Csapak <d.csapak@proxmox.com>'s feedback:
> - added Operation enum(r/w), as suggested by
> - added active operation tracking
> - combine type and message into on field
>
> v2:
> - check for maintenance now directly in lookup_datastore
> - parameter for checking is now the last acceptable maintenance type,
> description in commit msg of 2nd patch
> - ui cleanup
>
> Hannes Laimer (6):
> api-types: add maintenance type
> datastore: add check for maintenance in lookup
> pbs-datastore: add active operations tracking
> api: make maintenance_type updatable
> api: add get_active_operations endpoint
> ui: add option to change the maintenance type
>
> pbs-api-types/src/datastore.rs | 8 +-
> pbs-api-types/src/lib.rs | 3 +
> pbs-api-types/src/maintenance.rs | 83 +++++++++++++++++++
> pbs-datastore/Cargo.toml | 1 +
> pbs-datastore/src/datastore.rs | 119 +++++++++++++++++++--------
> pbs-datastore/src/lib.rs | 4 +
> pbs-datastore/src/snapshot_reader.rs | 6 +-
> pbs-datastore/src/task_tracking.rs | 110 +++++++++++++++++++++++++
> src/api2/admin/datastore.rs | 81 ++++++++++++------
> src/api2/backup/mod.rs | 4 +-
> src/api2/config/datastore.rs | 5 ++
> src/api2/reader/mod.rs | 6 +-
> src/api2/status.rs | 4 +-
> src/api2/tape/backup.rs | 6 +-
> src/api2/tape/restore.rs | 6 +-
> src/bin/proxmox-backup-api.rs | 1 +
> src/bin/proxmox-backup-proxy.rs | 6 +-
> src/server/mod.rs | 16 +++-
> src/server/prune_job.rs | 4 +-
> src/server/pull.rs | 4 +-
> src/server/verify_job.rs | 4 +-
> www/Makefile | 1 +
> www/Utils.js | 23 ++++++
> www/datastore/OptionView.js | 30 +++++++
> www/window/MaintenanceOptions.js | 72 ++++++++++++++++
> 25 files changed, 522 insertions(+), 85 deletions(-)
> create mode 100644 pbs-api-types/src/maintenance.rs
> create mode 100644 pbs-datastore/src/task_tracking.rs
> create mode 100644 www/window/MaintenanceOptions.js
>
prev parent reply other threads:[~2022-02-03 10:53 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-02 15:49 Hannes Laimer
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 1/6] api-types: add maintenance type Hannes Laimer
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 2/6] datastore: add check for maintenance in lookup Hannes Laimer
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 3/6] pbs-datastore: add active operations tracking Hannes Laimer
2022-02-03 10:53 ` Dominik Csapak
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 4/6] api: make maintenance_type updatable Hannes Laimer
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 5/6] api: add get_active_operations endpoint Hannes Laimer
2022-02-02 15:49 ` [pbs-devel] [PATCH proxmox-backup v6 6/6] ui: add option to change the maintenance type Hannes Laimer
2022-02-03 10:53 ` Dominik Csapak
2022-02-03 10:53 ` Dominik Csapak [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b6ec05aa-18f3-5da5-befb-753150444a00@proxmox.com \
--to=d.csapak@proxmox.com \
--cc=h.laimer@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox