all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH installer v3 00/20] fix #5536: implement post-(auto-)installation notification mechanism
@ 2024-08-21  9:40 Christoph Heiss
  2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 01/20] tree-wide: fix some typos Christoph Heiss
                   ` (20 more replies)
  0 siblings, 21 replies; 23+ messages in thread
From: Christoph Heiss @ 2024-08-21  9:40 UTC (permalink / raw)
  To: pve-devel

This implements a mechanism for post-installation "notifications" via a
POST request [0] when using the auto-installer.

It's implemented as a separate, small utility to facilitate separation
of concerns and make the information gathering easier by having it
isolated in one place.

Patches #1 through #5 are simply clean-ups, refactors, etc. that were
done along the way and can be applied independently.

Most interesting here will be patch #16, which adds the actual
implementation of the post-hook. (Bind-)mounting the installed host
system is done using the existing `proxmox-chroot` utility, and the HTTP
POST functionality can fortunately be re-used 1:1 from
`proxmox-fetch-answer`.

I've also included an example of how the JSON body (pretty-printed and
reduced some things for readability) of such a post-installation request
would look like below, for reference. 
Where applicable (and sensible), I have tried to align the format as
much as possible with 1) the format as used in the `fetch-answer` POST
request and 2) PVE's /nodes/<host>/status API endpoint.

Feedback on the post-hook information schema is of course also very much
appreciated!

It should be noted that some information like DMI is generally very
depended on the motherboard/firmware, on what information is actually
available and filled-in. So the contents are expected to vary wildly
between machines and may also be empty, as in the example below from a
VM.

Tested this with both PVE and PBS ISOs, with BIOS, UEFI w/ and w/o
SecureBoot. PMG did not (yet) have a auto-installation-capable release.
The only really product-specific code is the version detection in
`proxmox-post-hook`, which already handles all three products, so it
should work OOTB too.

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=5536

History
-------

v2: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064764.html
v1: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064580.html

Notable changes v2 -> v3:
  * dropped patch #11 "auto-installer: tests: replace manual panic!() 
    with assert_eq!()"
  * split out some preparatory changes into separate patches, based on
    Aaron's feedback
  * fixed bug in run env serialization w.r.t. secureboot state

Notable changes v1 -> v2:
  * dropped already applied patches & rebased on master
  * new fields; now includes ISO version, SecureBoot state, CPU and DMI 
    info 
  * product information was split into separate fields & expanded
  * boot mode information was split into separate fields
  * product version is now retrieved from the package using dpkg-query 
    directly
  * kernel version was split into separate fields
  * all disks and NICs are now included, a field indicates whether they
    are boot disk or management interface, respectively
  * some new cleanup/refactoring patches as noted on v1 
    (thanks Stefan for the in-depth review!)

Post notification example json
------------------------------

{
  "debian-version": "12.5",
  "product": {
    "fullname": "Proxmox VE",
    "short": "pve",
    "version": "8.2.2"
  },
  "iso": {
    "release": "8.2",
    "isorelease": "1"
  },
  "kernel-version": {
    "sysname": "Linux",
    "release": "6.8.4-2-pve",
    "version": "#1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z)",
    "machine": "x86_64"
  },
  "boot-info": {
    "mode": "efi",
    "secureboot": true
  },
  "cpu-info": {
    "cores": 4,
    "cpus": 4,
    "flags": "fpu vme [..]",
    "hvm": true,
    "model": "AMD Ryzen 7 3700X 8-Core Processor",
    "sockets": 1
  },
  "dmi": {
    "system": {
      "serial": "",
      "sku": "",
      "uuid": "b2fd1aa2-dc6e-4d8f-ad67-6dcc31984938",
      "name": "Standard PC (Q35 + ICH9, 2009)"
    },
    "baseboard": {},
    "chassis": {
      "asset_tag": "",
      "serial": ""
    }
  },
  "filesystem": "ext4",
  "fqdn": "host.domain",
  "machine-id": "b8737afea804482697ffe04db69c73d1",
  "disks": [
    {
      "size": 8589934592,
      "is-bootdisk": true,
      "udev-properties": {
        "DEVNAME": "/dev/vda",
	[..]
      }
    },
    {
      "size": 8589934592,
      "udev-properties": {
        "DEVNAME": "/dev/vdb",
	[..]
      }
    }
  ],
  "network-interfaces": [
    {
      "mac": "de:ad:ff:c2:63:5e",
      "address": "10.0.0.27/24",
      "is-management": true,
      "udev-properties": {
        "INTERFACE": "enp6s18",
	[..]
      }
    },
    {
      "mac": "de:ad:ff:1c:c9:01",
      "udev-properties": {
        "INTERFACE": "enp6s19",
	[..]
      }
    }
  ],
  "ssh-public-host-keys": {
    "ecdsa": "ecdsa-sha2-nistp256 [..] root@host.domain",
    "ed25519": "ssh-ed25519 [..] root@host.domain",
    "rsa": "ssh-rsa [..] root@host.domain"
  }
}

Git diffstat
------------

Christoph Heiss (20):
  tree-wide: fix some typos
  fetch-answer: partition: fix clippy warning
  low level: run env: ensure `secure_boot` property is dumped as int
  common: simplify filesystem type serializing & Display trait impl
  common: setup: serialize `target_hd` as string explicitly
  common: split out installer setup files loading functionality
  common: setup: deserialize `secure_boot` property from runtime env
  common: http: pass url by reference
  debian: strip unused library dependencies
  fetch-answer: move http-related code to gated module in
    installer-common
  tree-wide: convert some more crates to use workspace dependencies
  auto-install-assistant: replace `PathBuf` parameters with
    `AsRef<Path>`
  auto-installer: tests: simplify empty disks check
  auto-installer: tests: replace `PathBuf` parameters with `AsRef<Path>`
  auto-installer: move `SystemDMI` struct to common crate
  auto-installer: answer: factor out answer file reading into function
  auto-installer: udevinfo: introduce type alias for udev properties
  fix #5536: auto-installer: answer: add `posthook` section
  fix #5536: post-hook: add utility for sending notifications after
    auto-install
  unconfigured.sh: run proxmox-post-hook after successful auto-install

 Cargo.toml                                    |  11 +
 Makefile                                      |   8 +-
 Proxmox/Install/RunEnv.pm                     |   3 +-
 debian/control                                |   1 +
 debian/install                                |   1 +
 debian/rules                                  |  13 +
 proxmox-auto-install-assistant/Cargo.toml     |  14 +-
 proxmox-auto-install-assistant/src/main.rs    |  27 +-
 proxmox-auto-installer/Cargo.toml             |  15 +-
 proxmox-auto-installer/src/answer.rs          |  27 +-
 .../src/bin/proxmox-auto-installer.rs         |  13 +-
 proxmox-auto-installer/src/sysinfo.rs         |  51 +-
 proxmox-auto-installer/src/udevinfo.rs        |   8 +-
 proxmox-auto-installer/src/utils.rs           |  15 +-
 proxmox-auto-installer/tests/parse-answer.rs  |  47 +-
 proxmox-chroot/Cargo.toml                     |   8 +-
 proxmox-fetch-answer/Cargo.toml               |  17 +-
 .../src/fetch_plugins/http.rs                 | 100 +--
 .../src/fetch_plugins/partition.rs            |   2 +-
 proxmox-installer-common/Cargo.toml           |  26 +-
 proxmox-installer-common/src/disk_checks.rs   |   2 +-
 proxmox-installer-common/src/http.rs          |  94 +++
 proxmox-installer-common/src/lib.rs           |   4 +
 proxmox-installer-common/src/options.rs       | 121 +--
 proxmox-installer-common/src/setup.rs         | 109 +--
 proxmox-installer-common/src/sysinfo.rs       |  52 ++
 proxmox-installer-common/src/utils.rs         |   2 +
 proxmox-post-hook/Cargo.toml                  |  18 +
 proxmox-post-hook/src/main.rs                 | 784 ++++++++++++++++++
 proxmox-tui-installer/Cargo.toml              |   8 +-
 proxmox-tui-installer/src/setup.rs            |   2 +-
 unconfigured.sh                               |  11 +-
 32 files changed, 1219 insertions(+), 395 deletions(-)
 create mode 100644 proxmox-installer-common/src/http.rs
 create mode 100644 proxmox-installer-common/src/sysinfo.rs
 create mode 100644 proxmox-post-hook/Cargo.toml
 create mode 100644 proxmox-post-hook/src/main.rs

-- 
2.45.1



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2024-11-10 18:36 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-08-21  9:40 [pve-devel] [PATCH installer v3 00/20] fix #5536: implement post-(auto-)installation notification mechanism Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 01/20] tree-wide: fix some typos Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 02/20] fetch-answer: partition: fix clippy warning Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 03/20] low level: run env: ensure `secure_boot` property is dumped as int Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 04/20] common: simplify filesystem type serializing & Display trait impl Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 05/20] common: setup: serialize `target_hd` as string explicitly Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 06/20] common: split out installer setup files loading functionality Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 07/20] common: setup: deserialize `secure_boot` property from runtime env Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 08/20] common: http: pass url by reference Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 09/20] debian: strip unused library dependencies Christoph Heiss
2024-11-10 18:31   ` Thomas Lamprecht
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 10/20] fetch-answer: move http-related code to gated module in installer-common Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 11/20] tree-wide: convert some more crates to use workspace dependencies Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 12/20] auto-install-assistant: replace `PathBuf` parameters with `AsRef<Path>` Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 13/20] auto-installer: tests: simplify empty disks check Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 14/20] auto-installer: tests: replace `PathBuf` parameters with `AsRef<Path>` Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 15/20] auto-installer: move `SystemDMI` struct to common crate Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 16/20] auto-installer: answer: factor out answer file reading into function Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 17/20] auto-installer: udevinfo: introduce type alias for udev properties Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 18/20] fix #5536: auto-installer: answer: add `posthook` section Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 19/20] fix #5536: post-hook: add utility for sending notifications after auto-install Christoph Heiss
2024-08-21  9:40 ` [pve-devel] [PATCH installer v3 20/20] unconfigured.sh: run proxmox-post-hook after successful auto-install Christoph Heiss
2024-11-10 18:36 ` [pve-devel] partially-applied: [PATCH installer v3 00/20] fix #5536: implement post-(auto-)installation notification mechanism Thomas Lamprecht

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal