From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id AEFBC1FF09C for ; Mon, 21 Sep 2026 11:53:24 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 94FD9214E9; Mon, 21 Sep 2026 11:53:19 +0200 (CEST) From: Thomas Ellmenreich To: pdm-devel@lists.proxmox.com Subject: [RFC datacenter-manager/proxmox 0/5] log: allow finegrained control logging levels Date: Mon, 21 Sep 2026 11:51:53 +0200 Message-ID: <20260921095210.229315-2-t.ellmenreich@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1789984393153 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.589 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: ERNOYK66R5YSW6FHW7NGRF7XO77PD3VD X-Message-ID-Hash: ERNOYK66R5YSW6FHW7NGRF7XO77PD3VD X-MailFrom: t.ellmenreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Thomas Ellmenreich X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The core of this change is a simple reinterpretation of the logging environment variables that we are already using. Instead of parsing them as a single level filter [0], they will now be interpreted as env filters [1]. Since the latter can still parse the former, this change is backwards compatible and will not break with any logging environment variables that have been set. Implementation -------------- As Gabriel Goller proposed in this [2] Bugzilla enhancement, by using EnvFilters [2], we can have more granular control over the logging happening in the different Proxmox products. The first two commits do exactly this and can thus be applied by themselves. The last three add tests as well as utilities to better communicate log levels to use cases that don't directly integrate with tracing. proxmox 1/5: A simple reordering of the logging layers. This had to be done since EnvFilter is currently not clonable and can thus be applied only once. Previously, the global filter was being applied unnecessarily often, on all layers and then on the tracing_log layer as well. proxmox 2/5: Actual implementation of EnvFilters. proxmox 3/5: Introduction of tests for our EnvFilter, parser, and constructor. This was initially created for me to better understand EnvFilters, but then evolved into tests to ensure consistent functionality with our 'default_log_level'. proxmox 4/5: Make the Logger's 'init' function return the configuration used, so that it may be reused by other loggers or similar. datacenter-manager 5/5: Use the configuration returned by the 'init' function of the Logger to determine whether to set the REST server debug mode or not. Different Log Options --------------------- Combining the different options of the EnvFilter as well as our own default log level, we have the following cases for logging: 1. Env contains 'simple': Meaning that the environment variable contains a simple 'info'... This means that that level is taken as the expected one. 2. Env contains 'simple', and 'module': Means that the env variable contains a simple log level as well as zero or more module specific log levels. As expected the simple level is applied as a default and then for the specified modules the defined levels apply. 3. Env only contains 'module': If the env only contains a module (or multiple), that is interpreted by tracing as disabling the default logging and only enabling logging for the module/s 4. Env is empty: No logs are printed at all, everything is hidden. 5. Env is not set: This is the only time the default comes into play, by being set as the default log level. Open Questions -------------- - To me, the idea of storing a nested config and then returning it on 'init' seems a bit overengineered, but I could not find a better way. Especially when taking into consideration that, for this [3] follow up issue, relaying the exact log level might be necessary. - I had the idea that instead of having all users of the proxmox-log crate provide their own 'default_log_level' we could define that inside of the proxmox-log crate. By doing so, we could have the default change between DEBUG and INFO depending on if we are in a normal or release build. - Initially, I found case 3. of the different logging cases quite confusing and would have thought that our default should apply as default in that case. Unfortunately, when setting a default and then applying the EnvFilter string, tracing applies the modules as expected, but then also overrides the default as '', which means off. Notes for the Maintainer ------------------------ - For all tracing backed logging in Proxmox products to support EnvFilters, only patches 1 and 2 have to be applied. That said, for addressing other log related issues like [3], some way of inspecting current logging config, as done by the 'init' function (patch 4), should be made available. - Patch 4 contains a less desirable implementation of Clone for EnvFilters but since tracing_subscriber only includes a Clone implementation from 0.3.20 onwards, I was not able to find a better solution. Either the less desirable implementation of Clone is applied, a newer version of tracing_subscriber is packaged, patch 4 and 5 are skipped or a different way to return the config is found. [0]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.LevelFilter.html [1]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html [2]: https://bugzilla.proxmox.com/show_bug.cgi?id=6081 [3]: https://bugzilla.proxmox.com/show_bug.cgi?id=4646 proxmox: Thomas Ellmenreich (4): log: replace per layer filtering by one global filter log: replace simple level filter with env filter log: add tests to the logger log: return the logger configuration after initialisation proxmox-log/Cargo.toml | 2 +- proxmox-log/src/builder.rs | 491 +++++++++++++++++++++++--- proxmox-log/src/lib.rs | 22 +- proxmox-log/src/pve_task_formatter.rs | 2 +- 4 files changed, 454 insertions(+), 63 deletions(-) proxmox-datacenter-manager: Thomas Ellmenreich (1): api: set REST server debug level based on actual log level server/src/bin/proxmox-datacenter-api/main.rs | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) Summary over all repositories: 5 files changed, 459 insertions(+), 66 deletions(-) -- Generated by murpp 0.12.0