[PATCH datacenter-manager 0/6] fix #7639: task cache: consider running tasks when updating the cutoff timestamp

[Lukas Wagner <l.wagner@proxmox.com>]

At the moment, PDM only polls finished tasks from remote nodes.

Considering two tasks:
A: starts at 100, ends at 300
B: starts ad 150, ends at 200

If PDM polls the node at t = 250, it would see that B has finished and
update the cutoff timestamp for the next poll to 150. However, since A
has a starttime of 100, and the cutoff determines the minimum starttime
of tasks to retrieve from the remote via the API, PDM would never fetch
task A, leading to gaps in the task archive.

The solution is to fetch all tasks, running and finished, and then
update the cutoff with the oldest running active task in mind.

A side effect is that we now also return foreign (as in, not started by
PDM) running tasks from the API, which makes them appear in the UI as
well. This was always intended as a future extension anyways, so this
should be okay.

To avoid polled active tasks appearing as 'stuck' until the next regular poll,
we also lower the interval in which we poll foreign active tasks to 30 seconds.


NOTE to testers/reviewers:

An easy way to observe the previously missing tasks is to use the 'Guest
Console' as a task with manually controllable duration.

For instance:

  - Start console for VM 100 in the PVE Web UI
  - Start console for VM 101 in the PVE Web UI
   -> both start a task that runs as long as the browser actively shows the console
   
  - Close console for VM 101
  - Force-refresh the task view in PDM
   -> the console task for VM 101 should now appear (it is finished)
  - Close console for VM 100
  - Force-refresh the task view in PDM
   -> the console task does NOT show up, since the cutoff has already been set
      to the start time of the task for VM 101, which started *later*


Reported in the community forum:

https://forum.proxmox.com/threads/180317/


proxmox-datacenter-manager:

Lukas Wagner (6):
  pdm-api-types: add NativeUpid::node() convenience getter
  task cache: tests: allow to provide an explicit end time in 'task'
    helper
  task cache: tests: add get_cutoff helper
  task cache: tests: add 'make_cache' convenience helper
  fix #7639: task cache: consider running tasks when updating the cutoff
    timestamp
  task cache: poll known active tasks every 30 seconds

 lib/pdm-api-types/src/remote_upid.rs    |  10 +
 server/src/remote_tasks/refresh_task.rs |  13 +-
 server/src/remote_tasks/task_cache.rs   | 252 ++++++++++++++----------
 3 files changed, 159 insertions(+), 116 deletions(-)


Summary over all repositories:
  3 files changed, 159 insertions(+), 116 deletions(-)

-- 
Generated by murpp 0.12.0