From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8E122607E0 for ; Thu, 13 Aug 2020 10:29:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B8F89180F9 for ; Thu, 13 Aug 2020 10:29:26 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id EF81418070 for ; Thu, 13 Aug 2020 10:29:22 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B1E9C445DC for ; Thu, 13 Aug 2020 10:29:22 +0200 (CEST) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Thu, 13 Aug 2020 10:29:12 +0200 Message-Id: <20200813082921.28946-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.049 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods NO_DNS_FOR_FROM 0.379 Envelope sender has no MX or A DNS records RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: [pbs-devel] [PATCH proxmox-backup v3 0/9] improve syncjob handling X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Aug 2020 08:29:57 -0000 by saving the state in a separate file instead of parsing the task log should be faster and more correct, since we cannot phase out the last sync if too many tasks are started we have to do the same for all other tasks that can be scheduled changes from v2: * change upid_read_status to also return the endtime (parsed from the log) * use wrapper struct that holds the state and a lock (Job) * use that wrapper to hold the lock from loading until the worker is finshed * better comments and documentation * squashed commits for schedules * polling the abort_future in do_sync_jobs worker * fix a bug in 8/9 (Proxmox.Utils vs Promxox.Utils) changes from v1: * rebase on master (new Userid struct) * use an enum instead of a struct for jobstate since it better represents what we want (this way we do not have to save the starttime twice for example) * add an enum for TaskState and use that (instead of strings) (by doing this we can better parse the state for the syncview) * incorporate dietmar suggestsions * refactor do_sync_job first and extend after that * improve the state display in the gui * make some columsn smaller in the gui Dominik Csapak (9): server: change status of a task from a string to an enum server/worker_task: let upid_read_status also return the endtime config: add JobState helper api2/admin/sync: use JobState for faster access to state info api/{pull,sync}: refactor to do_sync_job api2/pull: extend do_sync_job to also handle schedule and jobstate syncjob: use do_sync_job also for scheduled sync jobs ui: syncjob: improve task text rendering ui: syncjob: make some columns smaller src/api2/admin/sync.rs | 76 ++++------ src/api2/config/sync.rs | 2 + src/api2/node/tasks.rs | 8 +- src/api2/pull.rs | 66 ++++++++ src/api2/types/mod.rs | 2 +- src/bin/proxmox-backup-api.rs | 1 + src/bin/proxmox-backup-proxy.rs | 83 ++-------- src/config.rs | 1 + src/config/jobstate.rs | 258 ++++++++++++++++++++++++++++++++ src/server/worker_task.rs | 111 ++++++++++---- www/config/SyncView.js | 30 +++- 11 files changed, 482 insertions(+), 156 deletions(-) create mode 100644 src/config/jobstate.rs -- 2.20.1