[PATCH http-server 1/1] apiserver: send TLS close notify at the end of streamed responses

["Michael Köppl" <m.koeppl@proxmox.com>]

A streamed response (e.g. for the single file restore download), has no
Content-Length nor chunked encoding, so the client can only tell where
the body ends by the connection being closed. The response accounced
'Connection: Keep-Alive' and, at the end of the stream, push_shutdown()
shut down the socket directly, bypassing the TLS layer. The client then
receives a TCP FIN without a TLS close notify [0].

Without a close notify, a client cannot tell a complete body from a
truncated one. OpenSSL 3 reports this as an unexpected EOF, which makes
curl fail with error 56 even though all data arrived. Even though
browsers usually accept this, this affects users who might use this for
scripted downloads from endpoints such as the file-restore one.

Do not keep streamed response alive. finish_response() then disconnects
the client once all data is written and, since commit 07e56cc ("fix
unexpected EOF for client when closing TLS session"),
client_do_disconnect() sends a close notify before shutting down the
socket. Drop the push_shutdown() call, as it would shut down the socket
before that if the write buffer is already empty.

[0] https://www.rfc-editor.org/info/rfc9846/#name-closure-alerts

Fixes: 389ad88 ("support streaming data form fh to client")
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
---
Ran into this while writing an E2E test that downloads a single file
from a PBS backup through the file-restore API endpoint with curl,
which failed with exit code 56 even though the file was complete.
I tested this manually with curl, as well as the GUI file restore. On a
3-node cluster, downloaded a single file from a backup on node 2
proxied through node 1, without and with this patch applied. Node 1
receives the stream completely and without error in both cases.

 src/PVE/APIServer/AnyEvent.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/PVE/APIServer/AnyEvent.pm b/src/PVE/APIServer/AnyEvent.pm
index 449b1c0..9258b6e 100644
--- a/src/PVE/APIServer/AnyEvent.pm
+++ b/src/PVE/APIServer/AnyEvent.pm
@@ -274,7 +274,6 @@ sub response_stream {
                     # write out any remaining data
                     $reqhdl->push_write($hdl->{rbuf}) if length($hdl->{rbuf}) > 0;
                     $hdl->{rbuf} = "";
-                    $reqhdl->push_shutdown();
                     $self->finish_response($reqstate);
                 }
             };
@@ -366,6 +365,8 @@ sub response {
         $resp->remove_header("Content-Length");
     }

+    $reqstate->{keep_alive} = 0 if $stream_fh;
+
     if ($reqstate->{keep_alive} > 0) {
         $resp->push_header('Connection' => 'Keep-Alive');
     } else {
--
2.47.3