public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 05/10] server/worker_task: write older tasks into archive file
Date: Fri, 25 Sep 2020 16:13:21 +0200	[thread overview]
Message-ID: <20200925141327.25024-9-d.csapak@proxmox.com> (raw)
In-Reply-To: <20200925141327.25024-1-d.csapak@proxmox.com>

instead of removing tasks beyond the 1000 that are in the index
write them into an archive file by appending them at the end
this way we can later still read them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/server/worker_task.rs | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/src/server/worker_task.rs b/src/server/worker_task.rs
index 9152609e..98047814 100644
--- a/src/server/worker_task.rs
+++ b/src/server/worker_task.rs
@@ -1,6 +1,6 @@
 use std::collections::HashMap;
 use std::fs::File;
-use std::io::{Read, BufRead, BufReader};
+use std::io::{Read, Write, BufRead, BufReader};
 use std::panic::UnwindSafe;
 use std::sync::atomic::{AtomicBool, Ordering};
 use std::sync::{Arc, Mutex};
@@ -32,6 +32,7 @@ pub const PROXMOX_BACKUP_TASK_DIR: &str = PROXMOX_BACKUP_TASK_DIR_M!();
 pub const PROXMOX_BACKUP_TASK_LOCK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/.active.lock");
 pub const PROXMOX_BACKUP_ACTIVE_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/active");
 pub const PROXMOX_BACKUP_INDEX_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/index");
+pub const PROXMOX_BACKUP_ARCHIVE_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/archive");
 
 const MAX_INDEX_TASKS: usize = 1000;
 
@@ -410,6 +411,19 @@ fn update_active_workers(new_upid: Option<&UPID>) -> Result<Vec<TaskListInfo>, E
             .group(backup_user.gid),
     )?;
 
+    if !finish_list.is_empty() && start > 0 {
+        match std::fs::OpenOptions::new().append(true).create(true).open(PROXMOX_BACKUP_ARCHIVE_TASK_FN) {
+            Ok(mut writer) => {
+                for info in &finish_list[0..start] {
+                    writer.write_all(render_task_line(&info).as_bytes())?;
+                }
+            },
+            Err(err) => bail!("could not write task archive - {}", err),
+        }
+
+        nix::unistd::chown(PROXMOX_BACKUP_ARCHIVE_TASK_FN, Some(backup_user.uid), Some(backup_user.gid))?;
+    }
+
     drop(lock);
 
     finish_list.append(&mut active_list);
-- 
2.20.1





  parent reply	other threads:[~2020-09-25 14:14 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-25 14:13 [pbs-devel] [PATCH proxmox/proxmox-backup/widget-toolkit] improve task list handling Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox 1/3] proxmox/tools/fs: add shared lock helper Dominik Csapak
2020-09-28  5:10   ` [pbs-devel] applied: " Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox 2/3] proxmox/tools/fs: create tmpfile helper Dominik Csapak
2020-09-28  5:10   ` [pbs-devel] applied: " Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox 3/3] proxmox/tools: add logrotate module Dominik Csapak
2020-09-28  5:12   ` Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 01/10] api2/node/tasks: move userfilter to function signature Dominik Csapak
2020-09-28  5:18   ` [pbs-devel] applied: " Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 02/10] server/worker_task: refactor locking of the task list Dominik Csapak
2020-09-28  5:28   ` Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 03/10] server/worker_task: factor out task list rendering Dominik Csapak
2020-09-28  5:31   ` [pbs-devel] applied: " Dietmar Maurer
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 04/10] server/worker_task: split task list file into two Dominik Csapak
2020-09-28  5:43   ` Dietmar Maurer
2020-09-25 14:13 ` Dominik Csapak [this message]
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 06/10] server/worker_task: add TaskListInfoIterator Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 07/10] api2/node/tasks: use TaskListInfoIterator instead of read_task_list Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 08/10] api2/status: use the TaskListInfoIterator here Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 09/10] server/worker_task: remove unecessary read_task_list Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH proxmox-backup 10/10] proxmox-backup-proxy: add task archive rotation Dominik Csapak
2020-09-25 14:13 ` [pbs-devel] [PATCH widget-toolkit 1/1] node/Tasks: improve scroller behaviour on datastore loading Dominik Csapak
2020-09-29  7:19   ` [pbs-devel] applied: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200925141327.25024-9-d.csapak@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal