From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 97E72946FD for ; Wed, 10 Apr 2024 16:30:39 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 796AA122BE for ; Wed, 10 Apr 2024 16:30:39 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 10 Apr 2024 16:30:38 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2A93343C2E for ; Wed, 10 Apr 2024 16:30:38 +0200 (CEST) From: Maximiliano Sandoval To: pbs-devel@lists.proxmox.com Date: Wed, 10 Apr 2024 16:30:31 +0200 Message-Id: <20240410143036.81807-1-m.sandoval@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.013 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox v3 0/5] Teach HTTP client how to decode deflate X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Apr 2024 14:30:39 -0000 This patch series adds support for decoding HTTP requests if they contain the Content-Encoding=deflate header. Deciding on a public API (and implementing it) to set this header is out-of-scope for this merge request a the moment of writing 🙈. Note that currently the proxmox-rest-server replies with deflated replies in the following functions: - handle_api_request: Called as part of Formated::handle_request and H2Service::handle_request. Always compresses if the client declares that supports deflate. - handle_unformatted_api_request: Called as part of Unformatted::handle_request. It decides whether to compress the contents same as before - simple_static_file_download: Called as part of handle_static_file_download which is called in ApiConfig::handle_request, this one decides whether to compress based on the fn extension_to_content_type which is a map format:should-be-compressed and whether the clients supports it. - chunked_static_file_download is called as part of handle_static_file_downloda same as before One thing that might be worth pointing out is that according to [1], when the client supports `Accept-Encoding=deflate`, the HTTP server can reply with `Content-Encoding=deflate` and encode the body using zlib, which means that it should be encoded with deflate with the zlib headers included. This is somewhat confusing but in short it means that one should use a zlib encoder rather than a deflate encoder. At the moment the server always compresses using deflate without the zlib headers, not a zlib encoder. [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding#deflate Differences from v2: - Split into multiple commits - More tests - Add builders for DeflateEndoder and DeflateDecoder - Both the deflate encoder and decoder where moved into a folder Maximiliano Sandoval (5): compression: deflate: Move encoder into a mod compression: deflate: add builder pattern compression: deflate: add a decoder compression: deflate: add test module http: teach the Client how to decode deflate content proxmox-compression/Cargo.toml | 3 +- .../src/{ => deflate}/compression.rs | 59 +++++-- .../src/deflate/decompression.rs | 141 +++++++++++++++ proxmox-compression/src/deflate/mod.rs | 167 ++++++++++++++++++ proxmox-compression/src/lib.rs | 4 +- proxmox-compression/src/zip.rs | 2 +- proxmox-http/Cargo.toml | 7 + proxmox-http/src/client/simple.rs | 65 ++++++- 8 files changed, 430 insertions(+), 18 deletions(-) rename proxmox-compression/src/{ => deflate}/compression.rs (83%) create mode 100644 proxmox-compression/src/deflate/decompression.rs create mode 100644 proxmox-compression/src/deflate/mod.rs -- 2.39.2