Re: [pmg-devel] [RFC pmg-log-tracker] do not assume constant year based on file index

[Stoiko Ivanov <s.ivanov@proxmox.com>]

On Mon, 3 Jan 2022 18:39:15 +0100 (CET)
Dietmar Maurer <dietmar@proxmox.com> wrote:

> > the old logic is flawed (but that can be said about parsing
> > traditional syslog timestamps (w/o year) in general) - it got worse
> > with the change in bullseye of rotating syslog weekly by default -
> > resulting in users losing one week of logs per day in the new year   
> 
> Daily syslog rotation is essential for high volume mail system. Else the log files gets far to big! 
> 
> So cant we simply change logfile rotation back to daily?

We can consider that - but for most bullseye installations the ship
has sailed (assuming that most users do not diff the configs when debian
asks them to and simply press 'keep my version')

Additionally the one year per log-file really seems brittle to me, or
rather the patch should be independent of the logrotation settings.

On the other hand - the volume of the logfiles does not bother me too much:
* since quite a long while /var/log/mail.log (in debian's standard
  logrotate config, which we do not touch) was always rotated weekly and
  contains all relevant logs for PMG (as an additional copy to what's
  already in /var/log/syslog) - and I think most users did not notice it
* users actually running high-volume sites quickly find out that they
  want/need to adapt their logrotation schedule 
* Finally I've seen a few posts in our support channels where users
  actively use weekly rotation and a keep of 32 to get more logs parsed
  and available in the GUI
* comparing the weekly logs of a fairly active site here (50k - 82k
  mails/month) - we get < 100 MB logs/week

My idea for the longer run would be to:
* add a dedicated log for pmg, which logs time in ISO8601 (or if this
  proves too expensive to pase in unix-timestamps) and maybe the hostname
* make retention configurable via GUI
* either suggest in a wiki-page/the reference documentation or actually by
  shipping a modified config file - to drop /var/log/mail.log (and in the
  long run maybe also the mail facility logging from /var/log/syslog)