From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <m.frank@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 6169E93AF9
for <pmg-devel@lists.proxmox.com>; Wed, 21 Feb 2024 09:24:26 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 421D412B74
for <pmg-devel@lists.proxmox.com>; Wed, 21 Feb 2024 09:23:56 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pmg-devel@lists.proxmox.com>; Wed, 21 Feb 2024 09:23:55 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id DEE4B4439E
for <pmg-devel@lists.proxmox.com>; Wed, 21 Feb 2024 09:23:54 +0100 (CET)
Message-ID: <714053c2-bbdb-4bc7-a37d-6d6a6eb80f11@proxmox.com>
Date: Wed, 21 Feb 2024 09:23:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Alexander Zeidler <a.zeidler@proxmox.com>, pmg-devel@lists.proxmox.com
References: <20240118145532.1362939-1-m.frank@proxmox.com>
<3e646e056537758e0b96fea51d4a8826b741c498.camel@proxmox.com>
Content-Language: en-US
From: Markus Frank <m.frank@proxmox.com>
In-Reply-To: <3e646e056537758e0b96fea51d4a8826b741c498.camel@proxmox.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.031 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
T_SCC_BODY_TEXT_LINE -0.01 -
Subject: Re: [pmg-devel] [PATCH pmg-api v3] config: adjust max_filters
calculation to reflect current memory usage
X-BeenThere: pmg-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Mail Gateway development discussion
<pmg-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pmg-devel>,
<mailto:pmg-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pmg-devel/>
List-Post: <mailto:pmg-devel@lists.proxmox.com>
List-Help: <mailto:pmg-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel>,
<mailto:pmg-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 21 Feb 2024 08:24:26 -0000
Hello,
thanks for looking into this.
On 2024-02-19 19:04, Alexander Zeidler wrote:
> On Thu, 2024-01-18 at 15:55 +0100, Markus Frank wrote:
>
>> my $max_servers = 5;
>> my $servermem = 120;
>> + my $base;
>> my $memory = physical_memory();
>> - my $add_servers = int(($memory - 512)/$servermem);
>> + if ($memory < 3840) {
>> + warn "low amount of system memory installed, recommended is 4+ GB\n"
>> + ."to prevent OOM kills, it is better to set max_filters manually\n";
>> + $base = $memory > 1536 ? 1024 : 512;
>> + } else {
>> + $base = 2816;
>> + $servermem = 150;
>> + }
>> + my $add_servers = int(($memory - $base)/$servermem);
>> $max_servers += $add_servers if $add_servers > 0;
>> $max_servers = 40 if $max_servers > 40;
> 1. Perhaps we also want to increase the current max_filters limit of 40
> minus 2?
What would be the improvement? 38 processes seem enough to me.
Could you maybe trigger this much process at the same time?
>
>
> 2. With the recommended 4GB setup, the patch now only returns
> max_filters = 10 instead of the previous 32 as marked in the table
> below.
Since 4GB is lowest recommended memory setup, 10 processes seem fine to me.
Also I do not think a 2-4 Core 4GB setup could reach more 10 processes at the same time.
I could only trigger 3-6 processes running simultaneously in my testing with a flood of emails.
>
> (table with temporarily removed 38-limit for the patch columns)
>
> [1] [2] [3] [4] [5]
> RAM *0.975= yet *120 left patch left
> 0.5 499 3 139 3 *120 139
> 1 998 7 158 7 *120 158
> 2 1996 15 196 11 *120 676
> 3 2995 23 235 19 *120 715
>
> 4 3993 32 153 ==> 10 *150 2493
> 8 7987 38 3427 37 *150 2437
> 16 15974 38 11414 90 *150 2474
> 32 31948 38 27388 197 *150 2398
> 64 63897 38 59337 410 *150 2397
>
Thanks for this calculation.
It shows that the remaining memory would be similar across
multiple memory configurations with this new calculation.
> [1] installed
> [2] ca. MemTotal according to /proc/meminfo, hence used for calculation
> [3] max_filters returned by current code
> [4] expected servermem used in code and table for calculation
> [5] (RAM * 0.975) - (max_filters * servermem)