[pve-devel] [PATCH installer v2 00/17] fix #5536: implement post-(auto-)installation notification mechanism

* [pve-devel] [PATCH installer v2 00/17] fix #5536: implement post-(auto-)installation notification mechanism
@ 2024-07-18 13:48 Christoph Heiss
  2024-07-18 13:48 ` [pve-devel] [PATCH installer v2 01/17] tree-wide: fix some typos Christoph Heiss
                   ` (18 more replies)
  0 siblings, 19 replies; 34+ messages in thread
From: Christoph Heiss @ 2024-07-18 13:48 UTC (permalink / raw)
  To: pve-devel

This implements a mechanism for post-installation "notifications" via a
POST request [0] when using the auto-installer.

It's implemented as a separate, small utility to facilitate separation
of concerns and make the information gathering easier by having it
isolated in one place.

Patches #1 through #3 are simply clean-ups, refactors, etc. that were
done along the way and can be applied independently.

Most interesting here will be patch #16, which adds the actual
implementation of the post-hook. (Bind-)mounting the installed host
system is done using the existing `proxmox-chroot` utility, and the HTTP
POST functionality can fortunately be re-used 1:1 from
`proxmox-fetch-answer`.

I've also included an example of how the JSON body (pretty-printed and
reduced some things for readability) of such a post-installation request
would look like below, for reference. 
Where applicable (and sensible), I have tried to align the format as
much as possible with 1) the format as used in the `fetch-answer` POST
request and 2) PVE's /nodes/<host>/status API endpoint.

Feedback on the post-hook information schema is of course also very much
appreciated!

It should be noted that some information like DMI is generally very
depended on the motherboard/firmware, on what information is actually
available and filled-in. So the contents are expected to vary wildly
between machines and may also be empty, as in the example below from a
VM.

Tested this with both PVE and PBS ISOs, PMG did not (yet) have a
release with an auto-installation capable ISO. The only really
product-specific code is the version detection in `proxmox-post-hook`,
which already handles all three products.

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=5536

previous revisions
------------------

v1: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064580.html

Notable changes v1 -> v2:
  * dropped already applied patches & rebased on master
  * new fields; now includes ISO version, SecureBoot state, CPU and DMI 
    info 
  * product information was split into separate fields & expanded
  * boot mode information was split into separate fields
  * product version is now retrieved from the package using dpkg-query 
    directly
  * kernel version was split into separate fields
  * all disks and NICs are now included, a field indicates whether they
    are boot disk or management interface, respectively
  * some new cleanup/refactoring patches as noted on v1 
    (thanks Stefan for the in-depth review!)

post notification example json
------------------------------

{
  "debian-version": "12.5",
  "product": {
    "fullname": "Proxmox VE",
    "short": "pve",
    "version": "8.2.2"
  },
  "iso": {
    "release": "8.2",
    "isorelease": "1"
  },
  "kernel-version": {
    "sysname": "Linux",
    "release": "6.8.4-2-pve",
    "version": "#1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z)",
    "machine": "x86_64"
  },
  "boot-info": {
    "mode": "efi",
    "secureboot": true
  },
  "cpu-info": {
    "cores": 4,
    "cpus": 4,
    "flags": "fpu vme [..]",
    "hvm": true,
    "model": "AMD Ryzen 7 3700X 8-Core Processor",
    "sockets": 1
  },
  "dmi": {
    "system": {
      "serial": "",
      "sku": "",
      "uuid": "b2fd1aa2-dc6e-4d8f-ad67-6dcc31984938",
      "name": "Standard PC (Q35 + ICH9, 2009)"
    },
    "baseboard": {},
    "chassis": {
      "asset_tag": "",
      "serial": ""
    }
  },
  "filesystem": "ext4",
  "fqdn": "host.domain",
  "machine-id": "b8737afea804482697ffe04db69c73d1",
  "disks": [
    {
      "size": 8589934592,
      "is-bootdisk": true,
      "udev-properties": {
        "DEVNAME": "/dev/vda",
	[..]
      }
    },
    {
      "size": 8589934592,
      "udev-properties": {
        "DEVNAME": "/dev/vdb",
	[..]
      }
    }
  ],
  "network-interfaces": [
    {
      "mac": "de:ad:ff:c2:63:5e",
      "address": "10.0.0.27/24",
      "is-management": true,
      "udev-properties": {
        "INTERFACE": "enp6s18",
	[..]
      }
    },
    {
      "mac": "de:ad:ff:1c:c9:01",
      "udev-properties": {
        "INTERFACE": "enp6s19",
	[..]
      }
    }
  ],
  "ssh-public-host-keys": {
    "ecdsa": "ecdsa-sha2-nistp256 [..] root@host.domain",
    "ed25519": "ssh-ed25519 [..] root@host.domain",
    "rsa": "ssh-rsa [..] root@host.domain"
  }
}

git shortlog
------------

Christoph Heiss (17):
  tree-wide: fix some typos
  fetch-answer: partition: fix clippy warning
  common: simplify filesystem type serializing & Display trait impl
  common: setup: serialize `target_hd` as string explicitly
  common: split out installer setup files loading functionality
  common: setup: deserialize `secure_boot` property from runtime env
  debian: strip unused library dependencies
  fetch-answer: move http-related code to gated module in
    installer-common
  tree-wide: convert some more crates to use workspace dependencies
  auto-install-assistant: replace `PathBuf` parameters with
    `AsRef<Path>`
  auto-installer: tests: replace manual panic!() with assert_eq!()
  auto-installer: tests: simplify empty disks check
  auto-installer: tests: replace `PathBuf` parameters with `AsRef<Path>`
  auto-installer: move `SystemDMI` struct to common crate
  fix #5536: auto-installer: answer: add `posthook` section
  fix #5536: post-hook: add utility for sending notifications after
    auto-install
  unconfigured.sh: run proxmox-post-hook after successful auto-install

 Cargo.toml                                    |  11 +
 Makefile                                      |   8 +-
 Proxmox/Install/RunEnv.pm                     |   1 +
 debian/control                                |   1 +
 debian/install                                |   1 +
 debian/rules                                  |  13 +
 proxmox-auto-install-assistant/Cargo.toml     |  14 +-
 proxmox-auto-install-assistant/src/main.rs    |  27 +-
 proxmox-auto-installer/Cargo.toml             |  15 +-
 proxmox-auto-installer/src/answer.rs          |  27 +-
 .../src/bin/proxmox-auto-installer.rs         |  13 +-
 proxmox-auto-installer/src/sysinfo.rs         |  51 +-
 proxmox-auto-installer/src/udevinfo.rs        |   8 +-
 proxmox-auto-installer/src/utils.rs           |  15 +-
 proxmox-auto-installer/tests/parse-answer.rs  |  55 +-
 proxmox-chroot/Cargo.toml                     |   8 +-
 proxmox-fetch-answer/Cargo.toml               |  17 +-
 .../src/fetch_plugins/http.rs                 | 100 +--
 .../src/fetch_plugins/partition.rs            |   2 +-
 proxmox-installer-common/Cargo.toml           |  26 +-
 proxmox-installer-common/src/disk_checks.rs   |   2 +-
 proxmox-installer-common/src/http.rs          |  94 +++
 proxmox-installer-common/src/lib.rs           |   4 +
 proxmox-installer-common/src/options.rs       | 121 +--
 proxmox-installer-common/src/setup.rs         | 109 +--
 proxmox-installer-common/src/sysinfo.rs       |  52 ++
 proxmox-installer-common/src/utils.rs         |   2 +
 proxmox-post-hook/Cargo.toml                  |  18 +
 proxmox-post-hook/src/main.rs                 | 784 ++++++++++++++++++
 proxmox-tui-installer/Cargo.toml              |   8 +-
 proxmox-tui-installer/src/setup.rs            |   2 +-
 unconfigured.sh                               |   7 +-
 32 files changed, 1218 insertions(+), 398 deletions(-)
 create mode 100644 proxmox-installer-common/src/http.rs
 create mode 100644 proxmox-installer-common/src/sysinfo.rs
 create mode 100644 proxmox-post-hook/Cargo.toml
 create mode 100644 proxmox-post-hook/src/main.rs

-- 
2.45.1

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

^ permalink raw reply	[flat|nested] 34+ messages in thread