public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Filip Schauer <f.schauer@proxmox.com>
To: Dietmar Maurer <dietmar@proxmox.com>,
	Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [PATCH common] tools: file_set_contents: use syswrite instead of print
Date: Wed, 25 Sep 2024 14:54:51 +0200	[thread overview]
Message-ID: <2c6b6b14-2f28-4a8b-bf2e-5576ba1d249c@proxmox.com> (raw)
In-Reply-To: <623295300.4070.1727253594226@webmail.proxmox.com>

I am appending the following to the commit message:

Since `file_set_contents` also performs a `rename` after writing, the
following table shows the results when the file is written without
renaming it afterwards:

            print                big_writes+print big_writes+syswrite
file size  written     amplif.  written     amplif.  written amplif.
     1 KiB      29 KiB     29.0      29 KiB     29.0     25 KiB      25.0
     2 KiB      29 KiB     14.5      30 KiB     15.0     25 KiB      12.5
     4 KiB      37 KiB      9.3      44 KiB     11.0     41 KiB      10.3
     8 KiB      61 KiB      7.6      45 KiB      5.6     45 KiB       5.6
    16 KiB     143 KiB      8.9      86 KiB      5.4     57 KiB       3.6
    32 KiB     396 KiB     12.4     225 KiB      7.0     69 KiB       2.2
    64 KiB    1281 KiB     20.0     673 KiB     10.5    105 KiB       1.6
   128 KiB    4789 KiB     37.4    3478 KiB     27.2    169 KiB       1.3
   256 KiB   18868 KiB     73.7    9976 KiB     39.0    572 KiB       2.2
   512 KiB   79304 KiB    154.9   42714 KiB     83.4   2150 KiB       4.2
  1024 KiB  347929 KiB    339.8  182483 KiB    178.2  11133 KiB      10.9

On 25/09/2024 10:39, Dietmar Maurer wrote:
> Please can you add the column showing write amplification using dd instead of file_set_contents, so that we can also see the minimal write amplf. from sqlite.
>
>> The table below illustrates the drastic reduction in write
>> amplification when writing files of different sizes to `/etc/pve/` using
>> `file_set_contents`:
>>
>>             print                big_writes+print     big_writes+syswrite
>> data size  written     amplif.  written     amplif.  written    amplif.
>>      1 KiB      48 KiB     48.0      45 KiB     45.0     41 KiB     41.0
>>      2 KiB      48 KiB     24.0      45 KiB     22.5     62 KiB     31.0
>>      4 KiB      82 KiB     20.5      80 KiB     20.0     73 KiB     18.3
>>      8 KiB     121 KiB     15.1      90 KiB     11.3     89 KiB     11.1
>>     16 KiB     217 KiB     13.6     146 KiB      9.1    113 KiB      7.1
>>     32 KiB     506 KiB     15.8     314 KiB      9.8    158 KiB      4.9
>>     64 KiB    1472 KiB     23.0     826 KiB     12.9    259 KiB      4.0
>>    128 KiB    5585 KiB     43.6    3765 KiB     29.4    452 KiB      3.5
>>    256 KiB   20424 KiB     79.8   10743 KiB     42.0   2351 KiB      9.2
>>    512 KiB   86715 KiB    169.4   43650 KiB     85.3   3204 KiB      6.3
>>   1024 KiB  369568 KiB    360.9  187496 KiB    183.1  15845 KiB     15.5
>>


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

  reply	other threads:[~2024-09-25 12:54 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-24 14:33 Filip Schauer
2024-09-25  8:39 ` Dietmar Maurer
2024-09-25 12:54   ` Filip Schauer [this message]
2024-09-25 14:37 ` Dominik Csapak
2024-09-30 11:38   ` Filip Schauer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2c6b6b14-2f28-4a8b-bf2e-5576ba1d249c@proxmox.com \
    --to=f.schauer@proxmox.com \
    --cc=dietmar@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal