From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 48E981FF0AF for ; Fri, 23 Oct 2026 13:31:39 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 3C310216C0; Fri, 23 Oct 2026 13:31:28 +0200 (CEST) From: Thomas Ellmenreich To: pdm-devel@lists.proxmox.com Subject: [PATCH datacenter-manager/proxmox v2 0/7] log: allow finegrained control log levels Date: Fri, 23 Oct 2026 13:31:11 +0200 Message-ID: <20261023113120.332453-1-t.ellmenreich@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1792755082230 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.386 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: E4WRNSGTD35WLPSKVRMVDGWK4LFC5ZKG X-Message-ID-Hash: E4WRNSGTD35WLPSKVRMVDGWK4LFC5ZKG X-MailFrom: t.ellmenreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Thomas Ellmenreich X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The core of this change is a simple reinterpretation of the logging environment variables that we are already using. Instead of parsing them as a single level filter [0], they will now be interpreted as env filters [1]. Since the latter can still parse all of the non erroring cases of the former, this change is backwards compatible and will not break with any logging environment variables that have been set. Implementation -------------- As Gabriel Goller proposed in this [2] Bugzilla enhancement, by using EnvFilters [1], we can have more granular control over the logging happening in the different Proxmox products. The first two commits do exactly this and can thus be applied by themselves. The rest of the commits include cleanup, tests, and some further improvements in rest server code that depended on the log variable. proxmox 1/7: A simple reordering of the logging layers. This had to be done since EnvFilter is currently not clonable and can thus be applied only once. Previously, the global filter was being applied unnecessarily often, on all layers and then on the tracing_log layer as well. proxmox 2/7: Actual implementation of EnvFilters. proxmox 3/7: Introduction of tests for our EnvFilter, parser, and constructor. This was initially created for me to better understand EnvFilters, but then evolved into tests to ensure consistent functionality with our 'default_log_level'. proxmox 4/7: Simple clean up of empty structs to unit structs and unnecessarily explicit module paths that clutter struct definitions. proxmox 5/7: Change the REST server default behaviour for setting its debug flag. Instead of defaulting to false, read the current log level and set it accordingly. datacenter-manager 6/7: The REST server now reads the current log level to set its debug flag, so we can remove the code doing the same thing from the datacenter-manager side. datacenter-manager 7/7: Replaced the PROXMOX_DEBUG environment variable by PDM_LOG, which is more in line with the other products. For both levels of api. Different Log Options --------------------- Combining the different options of the EnvFilter as well as our own default log level, we have the following cases for logging: 1. Env contains 'simple': Meaning that the environment variable contains a simple 'info'... This means that that level is taken as the expected one. 2. Env contains 'simple', and 'module': Means that the env variable contains a simple log level as well as zero or more module specific log levels. As expected the simple level is applied as a default and then for the specified modules the defined levels apply. 3. Env only contains 'module': If the env only contains a module (or multiple), that is interpreted by tracing as disabling the default logging and only enabling logging for the module/s 4. Env is empty: No logs are printed at all, everything is hidden. 5. Env is not set: This is the only time the default comes into play, by being set as the default log level. 6. Env is gibberish: An error is printed to stderr and the default log level is used. Notes for the Maintainer ------------------------ * To only apply the EnvFilter functionality the first two patches are the bare minimum, they restructure the filtering and then implement env filtering. * Patches 3 and 4 add tests and some cleanup respectivly. Neither adds any functionality and they can be applied indipendently. * Patches 5 and 6 should be applied together. No hard errors will appear if they are not applied together but Patch 6 epxects the REST server to set its own debug flag. * Patch 7 could maybe be delayed to after the next release as it is a breaking change. Future Work ----------- Since the REST server now sets its own debug flag according to the log level, other calls to the AcceptBuilder::debug() function can potentially be removed. This is the case for proxmox_debug that currently does exactly what the REST server will soon do by default. Changelog --------- - PATCH v1 -> v2 (thanks @Gabriel, @Lukas) * Also renamed the PROXMOX_DEBUG log variable into PDM_LOG for the privileged api, instead of only for the normal api. * I had previously removed the filter passed to the LogTracer, since LogTracer does not directly accept EnvFilter and the filter cannot be cloned. Now I get the max level in the EnvFilter and apply that as a LogTracer filter. * The REST server now checks the log level itself, so I removed the code setting the debug flag in proxmox-datacenter-manager. * Renamed the global_log_level to global_log_filter, as that is more fitting for a EnvFilter - RFC v2 -> PATCH v1 (thanks @Lukas) * Split the 'proxmox 1/4' commit into two commits. Instead of mixing the global filter mix with the cleanup, they are now two different commits. To make sure that the core patches can be applied independently, the cleanup commit was moved to 'proxmox 3/6'. * Fixed some clippy warnings relating to documentation indenting. * Added tests that also directly call 'tracing' instead of just calling 'log', just to confirm that both work and not just 'log'. * Introduced a new patch at the end, renaming the PROXMOX_DEBUG environment variable to PDM_LOG, as is default for all other products. - RFC v1 -> v2 * Removed patch 'v1 proxmox 4/5', which introduced a LoggerConfig returned by the Logger::init function. Deemed it unnecessary since I replaced the usage for it in patch 'v1 datacenter-manager 5/5' with a simple 'enabled!' and the idea of using it in the solution for [3] was abandoned. * Cleaned up this cover letter [0]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.LevelFilter.html [1]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html [2]: https://bugzilla.proxmox.com/show_bug.cgi?id=6081 [3]: https://bugzilla.proxmox.com/show_bug.cgi?id=4646 proxmox: Thomas Ellmenreich (5): log: replace per layer filtering with one global filter fix #6081: log: replace simple level filter with env filter log: add tests to the logger log: clean up of logger components rest-server: enable debug based on log level proxmox-log/Cargo.toml | 2 +- proxmox-log/src/builder.rs | 407 +++++++++++++++++++++++--- proxmox-log/src/lib.rs | 16 - proxmox-log/src/pve_task_formatter.rs | 2 +- proxmox-rest-server/src/connection.rs | 2 +- 5 files changed, 370 insertions(+), 59 deletions(-) proxmox-datacenter-manager: Thomas Ellmenreich (2): api: remove manual setting of REST server debug flag api: rename PROXMOX_DEBUG log variable to PDM_LOG server/src/bin/proxmox-datacenter-api/main.rs | 15 +++++++++------ .../src/bin/proxmox-datacenter-privileged-api.rs | 9 ++++++++- 2 files changed, 17 insertions(+), 7 deletions(-) Summary over all repositories: 7 files changed, 387 insertions(+), 66 deletions(-) -- Generated by murpp 0.12.0