[PATCH common,qemu-server,network,manager 0/10] mark secrets in schemas and redact them from the system report

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

The system report only requires Sys.Audit, but contains all guest configs
and the SDN configs verbatim. So it includes cloud-init passwords, in plain
text for Windows guests, and the API tokens of SDN IPAM and DNS
integrations. A report is often sent to support by someone who does not
own these credentials, for example a hosting provider renting out VMs.

This series adds a 'sensitive' flag for schema properties and uses it to
redact the report:

 - pve-common gets the flag, and helpers that replace the values of
   flagged properties in the raw text of key/value and section config
   files. Working on the raw text keeps invalid lines visible, which are
   often what support needs to see.
 - qemu-server flags the cloud-init password, and no longer shows it in
   the warning about an unparsable config line. pve-network flags the
   IPAM and DNS tokens.
 - The report redacts guest, storage and SDN IPAM/DNS configs through
   their schemas. In the whole report it also replaces passwords in URLs
   and values whose name suggests a credential, like mount options and
   container environment variables. The pvereport man page lists what
   gets redacted and where credentials can remain.

The flag only declares a value as secret; it changes neither validation
nor how values are stored. It can later also drive generic masking of API
output and task logs.

If it looks alright in general I can then add support for this also on
the rust side, partially prepared here already locally. And we can then
also leverage this more, like midterm also do a slightly more elaborate
secret-store concept with the biggest benefit being an unified API and
potentially different backends (pmxcfs, simple file store) and (at rest)
encryption and so on.

Repo (build-)dependencies:
 qemu-server, pve-network, pve-manager -> pve-common
The report change only has an effect with the qemu-server and pve-network
patches applied.

pve-common:

Thomas Lamprecht (4):
  section config: do not add isolated property lists on lookups
  schema: allow marking properties as sensitive
  schema: add helpers to redact sensitive values
  section config: add redaction of sensitive values in raw configs

 src/PVE/JSONSchema.pm                         |  51 ++++++
 src/PVE/SectionConfig.pm                      |  51 +++++-
 test/json-schema-test.pl                      | 148 ++++++++++++++++++
 .../section_config_property_isolation_test.pl |  52 ++++++
 test/section_config_test.pl                   |  40 +++++
 5 files changed, 341 insertions(+), 1 deletion(-)

qemu-server:

Thomas Lamprecht (2):
  cloud-init: mark the password as sensitive
  config: hide sensitive values of unparsable lines in warnings

 src/PVE/QemuServer.pm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

pve-network:

Thomas Lamprecht (1):
  sdn: ipam, dns: mark API tokens as sensitive

 src/PVE/Network/SDN/Dns/PowerdnsPlugin.pm  | 1 +
 src/PVE/Network/SDN/Ipams/PhpIpamPlugin.pm | 1 +
 2 files changed, 2 insertions(+)

pve-manager:

Thomas Lamprecht (3):
  report: redact sensitive values from guest, storage and SDN configs
  report: redact credentials in URLs, options and environment variables
  pvereport: man page: document which values get redacted

 PVE/Report.pm       | 78 ++++++++++++++++++++++++++++++++++++++++-----
 bin/pvereport       | 31 ++++++++++++++++++
 test/Makefile       |  6 +++-
 test/report_test.pl | 78 +++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 184 insertions(+), 9 deletions(-)
 create mode 100755 test/report_test.pl

-- 
2.47.3