From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	by lore.proxmox.com (Postfix) with ESMTPS id C854C1FF189
	for <inbox@lore.proxmox.com>; Fri,  4 Apr 2025 11:19:41 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 4610F1A84A;
	Fri,  4 Apr 2025 11:19:28 +0200 (CEST)
Message-ID: <9bf859f5-99a2-461a-9ad0-8d983474d2d0@proxmox.com>
Date: Fri, 4 Apr 2025 11:18:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Thomas Lamprecht <t.lamprecht@proxmox.com>,
 Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
 Maximiliano Sandoval <m.sandoval@proxmox.com>
References: <20250404075957.80057-1-f.weber@proxmox.com>
 <s8oa58w72hl.fsf@proxmox.com>
 <84420a11-0b5f-443f-99b3-bd517725a31c@proxmox.com>
Content-Language: en-US
From: Friedrich Weber <f.weber@proxmox.com>
In-Reply-To: <84420a11-0b5f-443f-99b3-bd517725a31c@proxmox.com>
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.010 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DMARC_MISSING             0.1 Missing DMARC policy
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [service.in, freedesktop.org]
Subject: Re: [pve-devel] [PATCH corosync] corosync.service: add patch to
 reduce log spam in broken network setups
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>

On 04/04/2025 10:55, Thomas Lamprecht wrote:
> Am 04.04.25 um 10:14 schrieb Maximiliano Sandoval:
>> Friedrich Weber <f.weber@proxmox.com> writes:
> 
>  ...
> 
>> An option that might require lower maintenance would be to ship a
>> service file override, e.g. at
>> /lib/systemd/system/corosync.service.d/set-log-rate-limit.conf with
>> contents:
>>
>> ```
>> [Service]
>> LogRateLimitIntervalSec=1s
>> LogRateLimitBurst=200
>> ```
>>
>> No strong feelings, it is just a matter of taste.
> 
> Would be more fitting if we did not package corosync our self, as is
> this integrated way would be fine to me. That sasid yours could be too.

Hmm, is this cut off?

> But ...
> 
>>
>>> + 1 file changed, 2 insertions(+)
>>> +
>>> +diff --git a/init/corosync.service.in b/init/corosync.service.in
>>> +index bd2a48a9..3d7ea2db 100644
>>> +--- a/init/corosync.service.in
>>> ++++ b/init/corosync.service.in
>>> +@@ -10,6 +10,8 @@ EnvironmentFile=-@INITCONFIGDIR@/corosync
>>> + ExecStart=@SBINDIR@/corosync -f $COROSYNC_OPTIONS
>>> + ExecStop=@SBINDIR@/corosync-cfgtool -H --force
>>> + Type=notify
>>> ++LogRateLimitIntervalSec=1s
>>> ++LogRateLimitBurst=200
>>
>> 200 hundred messages per second might be a bit too many. Since we are
>> not sure how many messages a unlucky user might see, I would suggest to
>> lower it a bit for the time being, 100 is a good round number.
>>
> 
> ... well, this is a core cluster service, having more available from a
> log burst is IMO really justified here.
> 
> That's also why I won't apply this patch for now, systemd already has
> default rate limiting for _very_ noisy stuff, it can also handle high
> log rates just fine and this only affects broken setups until they got
> fixed.

If I read the journald.conf docs [1] right, the default interval is 30s
and the burst value is 10000 multiplied by a factor depending on the
free disk space, I guess 4-6 on reasonable setups -- this is a lot of
messages, but as you mention probably fine for limiting really noisy
services. I was more thinking about this from a technical support
point-of-view, where I'd fear that having extreme corosync logspam over
days or weeks would cause the actually interesting stuff to be rotated
away more quickly than I'd like. :)

But as we have no idea how many broken setups are out there, this is all
somewhat hypothetical, so I'm also fine with not applying this -- if we
get many user reports seeing logspam I guess we can still do this.

[1]
https://www.freedesktop.org/software/systemd/man/latest/journald.conf.html#RateLimitIntervalSec=


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel