From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <h.laimer@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id C01CD63194
 for <pbs-devel@lists.proxmox.com>; Mon, 24 Jan 2022 13:34:14 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id B77D921F10
 for <pbs-devel@lists.proxmox.com>; Mon, 24 Jan 2022 13:33:44 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id 3D5D121EFF
 for <pbs-devel@lists.proxmox.com>; Mon, 24 Jan 2022 13:33:43 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1611A4614B
 for <pbs-devel@lists.proxmox.com>; Mon, 24 Jan 2022 13:33:43 +0100 (CET)
Message-ID: <6677dfb5-9f14-228b-8b70-218c6d8af74a@proxmox.com>
Date: Mon, 24 Jan 2022 13:33:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Content-Language: en-US
To: pbs-devel@lists.proxmox.com
References: <20220124123159.27086-1-h.laimer@proxmox.com>
From: Hannes Laimer <h.laimer@proxmox.com>
In-Reply-To: <20220124123159.27086-1-h.laimer@proxmox.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.044 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 NICE_REPLY_A           -0.001 Looks like a legit reply (A)
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [datastore.rs, restore.rs, proxmox-backup-proxy.rs,
 proxmox-backup-api.rs, status.rs, mod.rs, maintenance.rs, backup.rs, lib.rs,
 pull.rs]
Subject: Re: [pbs-devel] [PATCH proxmox-backup v5 0/8] closes #3071:
 maintenance mode for datastore
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
 <pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Mon, 24 Jan 2022 12:34:14 -0000

Adds maintenance mode and tracking of active reading/writing operations.
The maintenance mode prevents the start of new operations if the type of
operation they would perform on the datastore would conflict with the
maintenance type that is currently set. This check is performed when
lookup_datastore is called. Tasks only call this function once at the
beginning, therefore updating the maintenance type cannot interfere with
already running tasks.

active operations tracking:
Changed file layout to now also keep track of the pid and the counts of
operations that that pid started, like this it is possible to not count
operations that were started by a dead process, since they are also not
active anymore. Whenever the file is updated, also entries of dead
processes are removed. When the file is read, only entries of active
processes are counted.

The UI shows a spinner with the count of conflictintg tasks (the tasks
that were started before the maintenance type was updated) next to it.
As soon as all conflicting tasks are finished a checkmark appears.

Am 24.01.22 um 13:31 schrieb Hannes Laimer:
> v5:
>   - use simple struct and serde instead of manual parsing for file
>   - move tracking related stuff into new file (task_tracking.rs)
> 
> v4:
>   - clones are not also tracked
>   - use lockfile, instead of locking the file
>   - track pid of the process which started smth
>   - updating maintenance mode is now always possible
>   - add get_active_operations endpoint for datastore
>   - ui: show count of conflicting tasks (or checkmark if no conflicting
>       operations are active)
> 
>   v3, based on Dominik Csapak <d.csapak@proxmox.com>'s feedback:
>   - added Operation enum(r/w), as suggested by
>   - added active operation tracking
>   - combine type and message into on field
> 
> v2:
>   - check for maintenance now directly in lookup_datastore
>   - parameter for checking is now the last acceptable maintenance type,
>     description in commit msg of 2nd patch
>   - ui cleanup
> 
> 
> Dominik Csapak (1):
>    api: tape: fix datastore lookup operations
> 
> Hannes Laimer (6):
>    api-types: add maintenance type
>    datastore: add check for maintenance in lookup
>    pbs-datastore: add active operations tracking
>    api: make maintenance_type updatable
>    api: add get_active_operations endpoint
>    ui: add option to change the maintenance type
> 
> Thomas Lamprecht (1):
>    datastore: avoid tuple-match, use plain if
> 
>   pbs-api-types/src/datastore.rs       |   8 +-
>   pbs-api-types/src/lib.rs             |   3 +
>   pbs-api-types/src/maintenance.rs     |  83 +++++++++++++++++++
>   pbs-datastore/Cargo.toml             |   1 +
>   pbs-datastore/src/datastore.rs       | 119 +++++++++++++++++++--------
>   pbs-datastore/src/lib.rs             |   4 +
>   pbs-datastore/src/snapshot_reader.rs |   6 +-
>   pbs-datastore/src/task_tracking.rs   |  92 +++++++++++++++++++++
>   src/api2/admin/datastore.rs          |  82 ++++++++++++------
>   src/api2/backup/mod.rs               |   4 +-
>   src/api2/config/datastore.rs         |   5 ++
>   src/api2/reader/mod.rs               |   6 +-
>   src/api2/status.rs                   |   4 +-
>   src/api2/tape/backup.rs              |   6 +-
>   src/api2/tape/restore.rs             |   6 +-
>   src/bin/proxmox-backup-api.rs        |   1 +
>   src/bin/proxmox-backup-proxy.rs      |   6 +-
>   src/server/mod.rs                    |  16 +++-
>   src/server/prune_job.rs              |   4 +-
>   src/server/pull.rs                   |   4 +-
>   src/server/verify_job.rs             |   4 +-
>   www/Makefile                         |   1 +
>   www/Utils.js                         |  23 ++++++
>   www/datastore/OptionView.js          |  30 +++++++
>   www/window/MaintenanceOptions.js     |  72 ++++++++++++++++
>   25 files changed, 505 insertions(+), 85 deletions(-)
>   create mode 100644 pbs-api-types/src/maintenance.rs
>   create mode 100644 pbs-datastore/src/task_tracking.rs
>   create mode 100644 www/window/MaintenanceOptions.js
>