From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DF4F581753 for ; Wed, 24 Nov 2021 15:48:19 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D693A2174F for ; Wed, 24 Nov 2021 15:47:49 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id D32482172B for ; Wed, 24 Nov 2021 15:47:48 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id AC06146042 for ; Wed, 24 Nov 2021 15:47:48 +0100 (CET) From: Dominik Csapak To: pve-devel@lists.proxmox.com Date: Wed, 24 Nov 2021 15:47:45 +0100 Message-Id: <20211124144748.68687-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.202 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH mini-journalreader/http-server/manager] optimize journal api cal X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2021 14:48:19 -0000 this series changes the 'journal' api call to stream the journal to the client, which reduces the memory footprint of the worker processes for very big logs, the web ui may struggle (depending on browser/cpu/etc.). if that turns out to be a common problem, we can further optimize to pagination if necessary proxmox-mini-journalreader: Dominik Csapak (1): add '-j' flag to output json src/mini-journalreader.c | 66 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 3 deletions(-) http-server: Dominik Csapak (1): http-server: let the api call decide the content-encoding src/PVE/APIServer/AnyEvent.pm | 3 +++ 1 file changed, 3 insertions(+) pve-manager: Dominik Csapak (1): api: journal: stream the journal data to the client PVE/API2/Nodes.pm | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) -- 2.30.2