[pmg-devel] [RFC PATCH pmg-api] api: cluster status: add timeout to api call

[pmg-devel] [RFC PATCH pmg-api] api: cluster status: add timeout to api call

[Dominik Csapak]

we query each other host in the cluster for it's status, but if we reach
the 30s limit of the pmgproxy, we get a

 communication failure(0)

error in the gui. To prevent this, add a (much) shorter timeout to each
api call, which makes sure that the whole status call succeeds.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
sending as RFC because i am not sure if this is the best approach:
we could have very asymmetric http call times, which would maybe
all go through without timeout, but we cannot really know that
beforehand.

Another approach could be to give as much time as we have left (with
buffer) and subtract the time we took for each call. This would be a
problem if the first takes so long that we don't have any time left
for the other calls...

 src/PMG/API2/Cluster.pm | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/PMG/API2/Cluster.pm b/src/PMG/API2/Cluster.pm
index 84dafab..5e0a987 100644
--- a/src/PMG/API2/Cluster.pm
+++ b/src/PMG/API2/Cluster.pm
@@ -217,6 +217,9 @@ __PACKAGE__->register_method({
 	    $res = [ $ni ];
 	}
 
+	# 30 seconds minus 5 seconds buffer time, split evenly for the other hosts
+	my $timeout = 25 / (scalar(@$res) - 1);
+
 	my $rpcenv = PMG::RESTEnvironment->get();
         my $authuser = $rpcenv->get_user();
 	my $ticket = $rpcenv->get_ticket();
@@ -231,6 +234,7 @@ __PACKAGE__->register_method({
 			ticket => $ticket,
 			cookie_name => 'PMGAuthCookie',
 			host => $ni->{ip},
+			timeout => $timeout,
 			cached_fingerprints => {
 			    $ni->{fingerprint} => 1,
 			});
-- 
2.30.2

Re: [pmg-devel] [RFC PATCH pmg-api] api: cluster status: add timeout to api call

[Thomas Lamprecht]

Am 17/04/2023 um 13:40 schrieb Dominik Csapak:
> we query each other host in the cluster for it's status, but if we reach
> the 30s limit of the pmgproxy, we get a
> 
>  communication failure(0)
> 
> error in the gui. To prevent this, add a (much) shorter timeout to each
> api call, which makes sure that the whole status call succeeds.
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> sending as RFC because i am not sure if this is the best approach:
> we could have very asymmetric http call times, which would maybe
> all go through without timeout, but we cannot really know that
> beforehand.
> 
> Another approach could be to give as much time as we have left (with
> buffer) and subtract the time we took for each call. This would be a
> problem if the first takes so long that we don't have any time left
> for the other calls...

this would be nicer though, as it actually reflects what we want to control,
namely not exceeding the total timeout of 30 (minus some), rather than giving
each the same averaged piece.

A even better solution could be to do all request async and await them with a
single 28s timeout for all "futures", maybe using an AnyEvent->timer() for the
timeout, an AnyEvent::HTTP::http_get in a loop for requests and an
AnyEvent->condvar to await either. The most complexity might be re-doing the
finger print checking again via the AnyEvent::TLS context, but checking our
API http-server's proxy_request method it doesn't seem _that_ bad either.