[PATCH cluster 05/12] pmxcfs: server: report responses exceeding the maximum message size

[Daniel Kral <d.kral@proxmox.com>]

The shared memory ring buffer has a fixed maximum buffer size of
MAX_MSG_SIZE (currently 1 MiB), which is set by the pmxcfs' libqb-based
IPC client. If the maximium buffer size is exceeded, the QB ringbuffer
will fail to allocate the chunk for the response.

qb_ipcs_response_sendv() will fail with -EAGAIN, which will be logged by
pmxcfs as a rather confusing user error:

    [ipcs] crit: qb_ipcs_response_send: Resource temporarily unavailable
    [libqb] error: error receiving from setup sock (/dev/shm/qb-[...]/qb): Bad file descriptor (9)

and makes users of $PVE::Cluster::ipcc_send_rec{,_json}() fail with:

    ipcc_send_rec[13] failed: Transport endpoint is not connected

In huge cluster setups with many nodes, storages, and many thousands of
guests, the current MAX_MSG_SIZE of 1 MiB could be exceeded.

For example, in a test cluster with 10,000 stopped guests, the response
of CFS_IPC_GET_RRD_DUMP does already have 919 KiB, where many of the
runtime values are undefined (U) and therefore the response size would
be much longer if they were all running. Similar response sizes can be
reached with CFS_IPC_GET_GUEST_LIST and
CFS_IPC_GET_GUEST_CONFIG_PROPERTIES.

Report response messages that exceed the maximum message size in the
client and server to make the issue clearer to discern from other
possible errors, e.g. an unresponsive IPC server.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
 src/pmxcfs/server.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/src/pmxcfs/server.c b/src/pmxcfs/server.c
index abe1bee..54cd3aa 100644
--- a/src/pmxcfs/server.c
+++ b/src/pmxcfs/server.c
@@ -449,15 +449,23 @@ static int32_t s1_msg_process_fn(qb_ipcs_connection_t *c, void *data, size_t siz
 
     cfs_debug("process result %d", result);
 
+    int iov_len = 2;
+    struct iovec iov[iov_len];
+    struct qb_ipc_response_header res_header;
+    int32_t max_msg_size = qb_ipcs_connection_get_buffer_size(c);
+
+    if (sizeof(res_header) + outbuf->len > max_msg_size) {
+        cfs_critical(
+            "response for id %d is too large: %ld > %d", request_id, outbuf->len, max_msg_size
+        );
+        result = -EMSGSIZE;
+    }
+
     /* Do not send the response body in case of an error */
     if (result < 0) {
         g_string_truncate(outbuf, 0);
     }
 
-    int iov_len = 2;
-    struct iovec iov[iov_len];
-    struct qb_ipc_response_header res_header;
-
     res_header.id = request_id;
     res_header.size = sizeof(res_header) + outbuf->len;
     res_header.error = result;
-- 
2.47.3