[PATCH e2e-tests 10/38] lib: pveinstance: add helper to wait for qmeventd's cleanup

["Michael Köppl" <m.koeppl@proxmox.com>]

When a QEMU process exits, qmeventd runs 'qm cleanup' for the VM on
its own. If the VM runs again by the time the cleanup gets the config
lock, for example after a hibernate and start or a rollback to a
snapshot with RAM, the cleanup holds the lock for up to 30 seconds.
Operations on the VM within that timeframe fail with a lock timeout.

Add a helper that waits for the cleanup to finish, for tests that
need to change the VM right after such an operation.

Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
---
 Proxmox/Test/PVEInstance.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/Proxmox/Test/PVEInstance.pm b/Proxmox/Test/PVEInstance.pm
index 132a275..7e0c611 100644
--- a/Proxmox/Test/PVEInstance.pm
+++ b/Proxmox/Test/PVEInstance.pm
@@ -96,6 +96,24 @@ sub ct_exec {
     return $out // '';
 }
 
+# Wait until qmeventd has cleaned up after a QEMU process of VM $vmid that exited after $since, a
+# timestamp of the node's clock. Helps prevent a race for the config lock between the cleanup
+# process and the VM.
+sub wait_for_qemu_cleanup {
+    my ($self, $vmid, $since, $timeout) = @_;
+
+    poll_until(
+        sub {
+            my ($log) = $self->run_command("journalctl -u qmeventd --since \@$since -o cat");
+            return 0 if ($log // '') !~ m/^Starting cleanup for \Q$vmid\E$/m;
+            return !eval { $self->run_command("pgrep -f 'qm cleanup $vmid( |\$)'"); 1 };
+        },
+        timeout => $timeout // 60,
+        interval => 1,
+        description => "qmeventd to clean up after VM $vmid",
+    );
+}
+
 sub destroy_guest {
     my ($self, $type, $vmid) = @_;
     return $self->destroy_guest_on($self->get_nodename(), $type, $vmid);
-- 
2.47.3