From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E686361D11 for ; Mon, 28 Sep 2020 15:32:45 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B155B2EAB4 for ; Mon, 28 Sep 2020 15:32:15 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 825762EA7B for ; Mon, 28 Sep 2020 15:32:13 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 4D26A45806 for ; Mon, 28 Sep 2020 15:32:13 +0200 (CEST) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Mon, 28 Sep 2020 15:32:03 +0200 Message-Id: <20200928133212.409-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.165 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods NO_DNS_FOR_FROM 0.379 Envelope sender has no MX or A DNS records RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: [pbs-devel] [PATCH proxmox-backup v2 0/9] improve task list handling X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 13:32:46 -0000 this series extends the task handling in a way so that we can safely have more than 1000 tasks and properly filter and read them this also introduces a daily task to rotate the now existing task archive when it is over 500k up to maximum 20 files strictly speaking the widget-toolkit patch is not necessary, but makes the user interface a bit better to use changes from v1: * rebases on master * move the logrotate to proxmox-backup * use zstd in logrotate instead of gzip * TaskListInfoIterator now has an option to only return the 'active' tasks (this is a performance optimization) NOTE: i did not resend the widget-toolkit patch, but i would still recommend that it gets applied Dominik Csapak (9): tools: add logrotate module server/worker_task: refactor locking of the task list server/worker_task: split task list file into two server/worker_task: write older tasks into archive file server/worker_task: add TaskListInfoIterator api2/node/tasks: use TaskListInfoIterator instead of read_task_list api2/status: use the TaskListInfoIterator here server/worker_task: remove unecessary read_task_list proxmox-backup-proxy: add task archive rotation src/api2/node/tasks.rs | 52 +++--- src/api2/status.rs | 32 +++- src/bin/proxmox-backup-proxy.rs | 96 ++++++++++ src/server/worker_task.rs | 300 ++++++++++++++++++++++++-------- src/tools.rs | 1 + src/tools/logrotate.rs | 184 ++++++++++++++++++++ 6 files changed, 553 insertions(+), 112 deletions(-) create mode 100644 src/tools/logrotate.rs -- 2.20.1