From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Matthias Heiserer <m.heiserer@proxmox.com>
Subject: [pve-devel] applied: [PATCH http-server] file upload: don't calculate MD5
Date: Thu, 13 Apr 2023 12:52:52 +0200 [thread overview]
Message-ID: <9aa968be-f39d-e138-693d-376d420cda8b@proxmox.com> (raw)
In-Reply-To: <20230412142248.184475-1-m.heiserer@proxmox.com>
Am 12/04/2023 um 16:22 schrieb Matthias Heiserer:
> Until now, we calculated the MD5 hash of any uploaded file during the upload, regardless
> of whether the user chose to provide a hash sum and algorithm.
> The hash was only logged in the syslog.
>
> As the user can provide a hash algorithm and a checksum when uploading a file,
> which gets automatically checked (after the upload), this is not needed anymore.
> Instead, the file name is logged.
>
> Depending on the speed of the network and the cpu, upload speed or CPU usage might improve:
> All tests were made by uploading a 3.6GB iso from the PVE host to a local VM.
> First line is with md5, second without.
>
> no networklimit
> multipart upload complete (size: 3826831360B time: 20.310s rate: 179.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 16.169s rate: 225.72MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> 125MB/s network
> In this test, pveproxy worker used x % CPU during the upload. As you can see, the reduced CPU usage is noticable in slower networks.
> ~75% CPU: multipart upload complete (size: 3826831360B time: 30.764s rate: 118.63MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> ~60% CPU: multipart upload complete (size: 3826831360B time: 30.763s rate: 118.64MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> qemu64 cpu, no network limit
> multipart upload complete (size: 3826831360B time: 46.113s rate: 79.14MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 41.492s rate: 87.96MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> qemu64, -aes, 1 core, 0.7 cpu
> multipart upload complete (size: 3826831360B time: 79.875s rate: 45.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 66.364s rate: 54.99MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
>
> Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
> ---
> src/PVE/APIServer/AnyEvent.pm | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
>
applied, thanks!
prev parent reply other threads:[~2023-04-13 10:53 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-12 14:22 [pve-devel] " Matthias Heiserer
2023-04-13 10:52 ` Thomas Lamprecht [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9aa968be-f39d-e138-693d-376d420cda8b@proxmox.com \
--to=t.lamprecht@proxmox.com \
--cc=m.heiserer@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.