public inbox for pmg-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Markus Frank <m.frank@proxmox.com>
To: Alexander Zeidler <a.zeidler@proxmox.com>, pmg-devel@lists.proxmox.com
Subject: Re: [pmg-devel] [PATCH pmg-api v3] config: adjust max_filters calculation to reflect current memory usage
Date: Wed, 21 Feb 2024 09:23:54 +0100	[thread overview]
Message-ID: <714053c2-bbdb-4bc7-a37d-6d6a6eb80f11@proxmox.com> (raw)
In-Reply-To: <3e646e056537758e0b96fea51d4a8826b741c498.camel@proxmox.com>

Hello,

thanks for looking into this.

On  2024-02-19 19:04, Alexander Zeidler wrote:
> On Thu, 2024-01-18 at 15:55 +0100, Markus Frank wrote:
> 
>>       my $max_servers = 5;
>>       my $servermem = 120;
>> +    my $base;
>>       my $memory = physical_memory();
>> -    my $add_servers = int(($memory - 512)/$servermem);
>> +    if ($memory < 3840) {
>> +	warn "low amount of system memory installed, recommended is 4+ GB\n"
>> +	    ."to prevent OOM kills, it is better to set max_filters manually\n";
>> +	$base = $memory > 1536 ? 1024 : 512;
>> +    } else {
>> +	$base = 2816;
>> +	$servermem = 150;
>> +    }
>> +    my $add_servers = int(($memory - $base)/$servermem);
>>       $max_servers += $add_servers if $add_servers > 0;
>>       $max_servers = 40 if  $max_servers > 40;
> 1. Perhaps we also want to increase the current max_filters limit of 40
> minus 2?
What would be the improvement? 38 processes seem enough to me.
Could you maybe trigger this much process at the same time?
> 
> 
> 2. With the recommended 4GB setup, the patch now only returns
> max_filters = 10 instead of the previous 32 as marked in the table
> below.
Since 4GB is lowest recommended memory setup, 10 processes seem fine to me.
Also I do not think a 2-4 Core 4GB setup could reach more 10 processes at the same time.
I could only trigger 3-6 processes running simultaneously in my testing with a flood of emails.
> 
> (table with temporarily removed 38-limit for the patch columns)
> 
> [1]     [2]    [3]  [4]   [5]
> RAM *0.975=    yet *120  left     patch       left
> 0.5     499      3        139         3 *120   139
>    1     998      7        158         7 *120   158
>    2    1996     15        196        11 *120   676
>    3    2995     23        235        19 *120   715
> 
>    4    3993     32        153    ==> 10 *150  2493
>    8    7987     38       3427        37 *150  2437
>   16   15974     38      11414        90 *150  2474
>   32   31948     38      27388       197 *150  2398
>   64   63897     38      59337       410 *150  2397
> 
Thanks for this calculation.
It shows that the remaining memory would be similar across
multiple memory configurations with this new calculation.

> [1] installed
> [2] ca. MemTotal according to /proc/meminfo, hence used for calculation
> [3] max_filters returned by current code
> [4] expected servermem used in code and table for calculation
> [5] (RAM * 0.975) - (max_filters * servermem)




  reply	other threads:[~2024-02-21  8:24 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-18 14:55 Markus Frank
2024-02-19 18:04 ` Alexander Zeidler
2024-02-21  8:23   ` Markus Frank [this message]
2024-02-21 16:28 ` [pmg-devel] applied: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=714053c2-bbdb-4bc7-a37d-6d6a6eb80f11@proxmox.com \
    --to=m.frank@proxmox.com \
    --cc=a.zeidler@proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal