public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v2 4/9] server/worker_task: write older tasks into archive file
Date: Mon, 28 Sep 2020 15:32:07 +0200	[thread overview]
Message-ID: <20200928133212.409-5-d.csapak@proxmox.com> (raw)
In-Reply-To: <20200928133212.409-1-d.csapak@proxmox.com>

instead of removing tasks beyond the 1000 that are in the index
write them into an archive file by appending them at the end
this way we can later still read them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/server/worker_task.rs | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/src/server/worker_task.rs b/src/server/worker_task.rs
index 2ce71136..4a4406e1 100644
--- a/src/server/worker_task.rs
+++ b/src/server/worker_task.rs
@@ -1,6 +1,6 @@
 use std::collections::HashMap;
 use std::fs::File;
-use std::io::{Read, BufRead, BufReader};
+use std::io::{Read, Write, BufRead, BufReader};
 use std::panic::UnwindSafe;
 use std::sync::atomic::{AtomicBool, Ordering};
 use std::sync::{Arc, Mutex};
@@ -32,6 +32,7 @@ pub const PROXMOX_BACKUP_TASK_DIR: &str = PROXMOX_BACKUP_TASK_DIR_M!();
 pub const PROXMOX_BACKUP_TASK_LOCK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/.active.lock");
 pub const PROXMOX_BACKUP_ACTIVE_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/active");
 pub const PROXMOX_BACKUP_INDEX_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/index");
+pub const PROXMOX_BACKUP_ARCHIVE_TASK_FN: &str = concat!(PROXMOX_BACKUP_TASK_DIR_M!(), "/archive");
 
 const MAX_INDEX_TASKS: usize = 1000;
 
@@ -407,6 +408,19 @@ fn update_active_workers(new_upid: Option<&UPID>) -> Result<Vec<TaskListInfo>, E
             .group(backup_user.gid),
     )?;
 
+    if !finish_list.is_empty() && start > 0 {
+        match std::fs::OpenOptions::new().append(true).create(true).open(PROXMOX_BACKUP_ARCHIVE_TASK_FN) {
+            Ok(mut writer) => {
+                for info in &finish_list[0..start] {
+                    writer.write_all(render_task_line(&info).as_bytes())?;
+                }
+            },
+            Err(err) => bail!("could not write task archive - {}", err),
+        }
+
+        nix::unistd::chown(PROXMOX_BACKUP_ARCHIVE_TASK_FN, Some(backup_user.uid), Some(backup_user.gid))?;
+    }
+
     drop(lock);
 
     finish_list.append(&mut active_list);
-- 
2.20.1





  parent reply	other threads:[~2020-09-28 13:32 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-28 13:32 [pbs-devel] [PATCH proxmox-backup v2 0/9] improve task list handling Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 1/9] tools: add logrotate module Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 2/9] server/worker_task: refactor locking of the task list Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 3/9] server/worker_task: split task list file into two Dominik Csapak
2020-09-28 13:32 ` Dominik Csapak [this message]
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 5/9] server/worker_task: add TaskListInfoIterator Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 6/9] api2/node/tasks: use TaskListInfoIterator instead of read_task_list Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 7/9] api2/status: use the TaskListInfoIterator here Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 8/9] server/worker_task: remove unecessary read_task_list Dominik Csapak
2020-09-28 13:32 ` [pbs-devel] [PATCH proxmox-backup v2 9/9] proxmox-backup-proxy: add task archive rotation Dominik Csapak
2020-09-29  7:16 ` [pbs-devel] applied: [PATCH proxmox-backup v2 0/9] improve task list handling Dietmar Maurer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200928133212.409-5-d.csapak@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal