public inbox for pdm-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Lukas Wagner" <l.wagner@proxmox.com>
To: "Dominik Csapak" <d.csapak@proxmox.com>,
	"Lukas Wagner" <l.wagner@proxmox.com>,
	<pdm-devel@lists.proxmox.com>
Subject: Re: [PATCH datacenter-manager 04/15] task cache: rotate: align timestamp for new files to UTC midnight
Date: Fri, 10 Jul 2026 14:02:38 +0200	[thread overview]
Message-ID: <DJUVJPUKAXDB.1NWS66XAYTQIN@proxmox.com> (raw)
In-Reply-To: <655cfb23-835d-456d-a2f8-5bc4ae0f7c07@proxmox.com>

On Fri Jul 10, 2026 at 1:36 PM CEST, Dominik Csapak wrote:
> wouldn't it make more sense to put the alignment inside rotate? so 
> potential future callers can't miss this?

That would of course also be possible, but semantically I think it fits
better in the caller. It does not really make a big practical difference
for us here, but if the task_cache module was reused somewhere else, a
different callers might have different expectations when it comes to
alignment (example, maybe prefer midnight in the local timezone, as a
trivial, maybe slightly synthetic example).
Not saying that putting the alignment into 'rotate/init' would be wrong,
just explaining my thinking here.

>
> also the init cache function does not align the timestamp
> so it's aligned only after rotation, is that intended?

Just an oversight, actually. I'll add the alignment to the init call as
well.

Thanks!

>
> On 7/2/26 11:23 AM, Lukas Wagner wrote:
>> This also requires to change the rotation check to an inclusive >=,
>> otherwise we would only rotate after two days instead of every day.
>> 
>> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
>> ---
>>   server/src/remote_tasks/refresh_task.rs | 23 ++++++++++++++++++++++-
>>   server/src/remote_tasks/task_cache.rs   |  3 ++-
>>   2 files changed, 24 insertions(+), 2 deletions(-)
>> 
>> diff --git a/server/src/remote_tasks/refresh_task.rs b/server/src/remote_tasks/refresh_task.rs
>> index 68ef5a25..55d5694e 100644
>> --- a/server/src/remote_tasks/refresh_task.rs
>> +++ b/server/src/remote_tasks/refresh_task.rs
>> @@ -360,7 +360,13 @@ fn get_remotes_with_finished_tasks(
>>   ///
>>   /// Returns Ok(true) the cache's files were rotated.
>>   async fn rotate_cache(cache: TaskCache) -> Result<bool, Error> {
>> -    tokio::task::spawn_blocking(move || cache.write()?.rotate(proxmox_time::epoch_i64())).await?
>> +    tokio::task::spawn_blocking(move || {
>> +        cache.write()?.rotate(align_timestamp(
>> +            proxmox_time::epoch_i64(),
>> +            super::ROTATE_AFTER as i64,
>> +        ))
>> +    })
>> +    .await?
>>   }
>>   
>>   /// Apply the task cache journal.
>> @@ -541,3 +547,18 @@ async fn update_task_cache(
>>       })
>>       .await?
>>   }
>> +
>> +/// Align a UNIX timestamp to the specified interval.
>> +fn align_timestamp(now: i64, interval: i64) -> i64 {
>> +    now - (now.rem_euclid(interval))
>> +}
>> +
>> +#[cfg(test)]
>> +mod tests {
>> +    use crate::remote_tasks::refresh_task::align_timestamp;
>> +
>> +    #[test]
>> +    fn test_align_timestamp() {
>> +        assert_eq!(align_timestamp(1782898009, 24 * 3600), 1782864000);
>> +    }
>> +}
>> diff --git a/server/src/remote_tasks/task_cache.rs b/server/src/remote_tasks/task_cache.rs
>> index b5fc3b21..55aa8b70 100644
>> --- a/server/src/remote_tasks/task_cache.rs
>> +++ b/server/src/remote_tasks/task_cache.rs
>> @@ -285,7 +285,8 @@ impl WritableTaskCache {
>>   
>>           match archive_files.first() {
>>               Some(bound) => {
>> -                if now > bound.starttime && now - bound.starttime > self.cache.rotate_after as i64 {
>> +                if now > bound.starttime && now - bound.starttime >= self.cache.rotate_after as i64
>> +                {
>>                       start_new_file(&mut archive_files)?;
>>                   }
>>               }





  reply	other threads:[~2026-07-10 12:03 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-02  9:22 [PATCH datacenter-manager/proxmox 00/15] task cache improvements (archive corruption handling, error handling) Lukas Wagner
2026-07-02  9:22 ` [PATCH proxmox 01/15] sys: fs: don't replace file extension make_tmp_file Lukas Wagner
2026-07-02  9:29   ` Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 02/15] task cache: fix missing cutoff state for PBS remotes Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 03/15] task cache: refresh task: don't apply journal if the archive was rotated Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 04/15] task cache: rotate: align timestamp for new files to UTC midnight Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-10 12:02     ` Lukas Wagner [this message]
2026-07-02  9:22 ` [PATCH datacenter-manager 05/15] task cache: add test case for task cache rotation Lukas Wagner
2026-07-10 11:35   ` Dominik Csapak
2026-07-10 12:13     ` Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 06/15] task cache: pre-compute static paths during initialization Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 07/15] task cache: only initialize `TaskCache` struct once Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 08/15] task cache: archive iterator: don't yield more items if reading from file failed Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-10 12:36     ` Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 09/15] task cache: include archive file path in error log messages Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-10 12:53     ` Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 10/15] task cache: introduce ArchiveFileWriter Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-02  9:22 ` [PATCH datacenter-manager 11/15] task cache: use ArchiveFileWriter when creating new archive files Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 12/15] task cache: trigger repair of corruption when applying journal Lukas Wagner
2026-07-02  9:22 ` [PATCH datacenter-manager 13/15] task cache: trigger repair of corruption when compressing archive files Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-02  9:22 ` [PATCH datacenter-manager 14/15] task cache: trigger repair of corruption after read-accesses Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-02  9:22 ` [PATCH datacenter-manager 15/15] task cache: handle potentially duplicated archive files after 'compress_archive_file' Lukas Wagner
2026-07-10 11:36   ` Dominik Csapak
2026-07-10 11:36 ` [PATCH datacenter-manager/proxmox 00/15] task cache improvements (archive corruption handling, error handling) Dominik Csapak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DJUVJPUKAXDB.1NWS66XAYTQIN@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=d.csapak@proxmox.com \
    --cc=pdm-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal