From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Matthias Heiserer <m.heiserer@proxmox.com>
Subject: [pve-devel] applied: [PATCH http-server] file upload: don't calculate MD5
Date: Thu, 13 Apr 2023 12:52:52 +0200 [thread overview]
Message-ID: <9aa968be-f39d-e138-693d-376d420cda8b@proxmox.com> (raw)
In-Reply-To: <20230412142248.184475-1-m.heiserer@proxmox.com>
Am 12/04/2023 um 16:22 schrieb Matthias Heiserer:
> Until now, we calculated the MD5 hash of any uploaded file during the upload, regardless
> of whether the user chose to provide a hash sum and algorithm.
> The hash was only logged in the syslog.
>
> As the user can provide a hash algorithm and a checksum when uploading a file,
> which gets automatically checked (after the upload), this is not needed anymore.
> Instead, the file name is logged.
>
> Depending on the speed of the network and the cpu, upload speed or CPU usage might improve:
> All tests were made by uploading a 3.6GB iso from the PVE host to a local VM.
> First line is with md5, second without.
>
> no networklimit
> multipart upload complete (size: 3826831360B time: 20.310s rate: 179.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 16.169s rate: 225.72MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> 125MB/s network
> In this test, pveproxy worker used x % CPU during the upload. As you can see, the reduced CPU usage is noticable in slower networks.
> ~75% CPU: multipart upload complete (size: 3826831360B time: 30.764s rate: 118.63MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> ~60% CPU: multipart upload complete (size: 3826831360B time: 30.763s rate: 118.64MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> qemu64 cpu, no network limit
> multipart upload complete (size: 3826831360B time: 46.113s rate: 79.14MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 41.492s rate: 87.96MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> qemu64, -aes, 1 core, 0.7 cpu
> multipart upload complete (size: 3826831360B time: 79.875s rate: 45.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 66.364s rate: 54.99MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
> ---
> src/PVE/APIServer/AnyEvent.pm | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
>
applied, thanks!
prev parent reply other threads:[~2023-04-13 10:53 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-12 14:22 [pve-devel] " Matthias Heiserer
2023-04-13 10:52 ` Thomas Lamprecht [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9aa968be-f39d-e138-693d-376d420cda8b@proxmox.com \
--to=t.lamprecht@proxmox.com \
--cc=m.heiserer@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox