From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 965361FF173 for ; Mon, 11 Nov 2024 14:16:18 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 121DBB40B; Mon, 11 Nov 2024 14:16:02 +0100 (CET) From: Christoph Heiss To: pve-devel@lists.proxmox.com Date: Mon, 11 Nov 2024 14:14:56 +0100 Message-ID: <20241111131519.867887-1-c.heiss@proxmox.com> X-Mailer: git-send-email 2.47.0 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.120 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH installer v4 00/12] fix #5536: implement post-(auto-)installation notification mechanism X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" This implements a mechanism for post-installation "notifications" via a POST request [0] when using the auto-installer. It's implemented as a separate, small utility to facilitate separation of concerns and make the information gathering easier by having it isolated in one place. Patches #1 through #5 are simply clean-ups, refactors, etc. that were done along the way and can be applied independently. Most interesting here will be patch #16, which adds the actual implementation of the post-hook. (Bind-)mounting the installed host system is done using the existing `proxmox-chroot` utility, and the HTTP POST functionality can fortunately be re-used 1:1 from `proxmox-fetch-answer`. I've also included an example of how the JSON body (pretty-printed and reduced some things for readability) of such a post-installation request would look like below, for reference. Where applicable (and sensible), I have tried to align the format as much as possible with 1) the format as used in the `fetch-answer` POST request and 2) PVE's /nodes//status API endpoint. Feedback on the post-hook information schema is of course also very much appreciated! It should be noted that some information like DMI is generally very depended on the motherboard/firmware, on what information is actually available and filled-in. So the contents are expected to vary wildly between machines and may also be empty, as in the example below from a VM. Tested this with both PVE and PBS ISOs, with BIOS, UEFI w/ and w/o SecureBoot. PMG did not (yet) have a auto-installation-capable release. The only really product-specific code is the version detection in `proxmox-post-hook`, which already handles all three products, so it should work OOTB too. [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5536 History ======= v3: https://lore.proxmox.com/pve-devel/20240821094023.667806-1-c.heiss@proxmox.com/ v2: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064764.html v1: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064580.html Notable changes v3 -> v4: * rebased on latest master * dropped already applied patches * use script from proxmox-backup for stripping unneeded dependencies instead of open-coding Notable changes v2 -> v3: * dropped patch #11 "auto-installer: tests: replace manual panic!() with assert_eq!()" * split out some preparatory changes into separate patches, based on Aaron's feedback * fixed bug in run env serialization w.r.t. secureboot state Notable changes v1 -> v2: * dropped already applied patches & rebased on master * new fields; now includes ISO version, SecureBoot state, CPU and DMI info * product information was split into separate fields & expanded * boot mode information was split into separate fields * product version is now retrieved from the package using dpkg-query directly * kernel version was split into separate fields * all disks and NICs are now included, a field indicates whether they are boot disk or management interface, respectively * some new cleanup/refactoring patches as noted on v1 (thanks Stefan for the in-depth review!) Post notification example json ============================== { "debian-version": "12.5", "product": { "fullname": "Proxmox VE", "short": "pve", "version": "8.2.2" }, "iso": { "release": "8.2", "isorelease": "1" }, "kernel-version": { "sysname": "Linux", "release": "6.8.4-2-pve", "version": "#1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z)", "machine": "x86_64" }, "boot-info": { "mode": "efi", "secureboot": true }, "cpu-info": { "cores": 4, "cpus": 4, "flags": "fpu vme [..]", "hvm": true, "model": "AMD Ryzen 7 3700X 8-Core Processor", "sockets": 1 }, "dmi": { "system": { "serial": "", "sku": "", "uuid": "b2fd1aa2-dc6e-4d8f-ad67-6dcc31984938", "name": "Standard PC (Q35 + ICH9, 2009)" }, "baseboard": {}, "chassis": { "asset_tag": "", "serial": "" } }, "filesystem": "ext4", "fqdn": "host.domain", "machine-id": "b8737afea804482697ffe04db69c73d1", "disks": [ { "size": 8589934592, "is-bootdisk": true, "udev-properties": { "DEVNAME": "/dev/vda", [..] } }, { "size": 8589934592, "udev-properties": { "DEVNAME": "/dev/vdb", [..] } } ], "network-interfaces": [ { "mac": "de:ad:ff:c2:63:5e", "address": "10.0.0.27/24", "is-management": true, "udev-properties": { "INTERFACE": "enp6s18", [..] } }, { "mac": "de:ad:ff:1c:c9:01", "udev-properties": { "INTERFACE": "enp6s19", [..] } } ], "ssh-public-host-keys": { "ecdsa": "ecdsa-sha2-nistp256 [..] root@host.domain", "ed25519": "ssh-ed25519 [..] root@host.domain", "rsa": "ssh-rsa [..] root@host.domain" } } Diffstat ======== Christoph Heiss (12): debian: strip unused library dependencies fetch-answer: move http-related code to gated module in installer-common tree-wide: convert some more crates to use workspace dependencies auto-install-assistant: replace `PathBuf` parameters with `AsRef` auto-installer: tests: simplify empty disks check auto-installer: tests: replace `PathBuf` parameters with `AsRef` auto-installer: move `SystemDMI` struct to common crate auto-installer: answer: factor out answer file reading into function auto-installer: udevinfo: introduce type alias for udev properties fix #5536: auto-installer: answer: add `posthook` section fix #5536: post-hook: add utility for sending notifications after auto-install unconfigured.sh: run proxmox-post-hook after successful auto-install Cargo.toml | 9 + Makefile | 8 +- debian/control | 1 + debian/install | 1 + debian/rules | 9 + .../scripts/elf-strip-unused-dependencies.sh | 20 + proxmox-auto-install-assistant/Cargo.toml | 13 +- proxmox-auto-install-assistant/src/main.rs | 23 +- proxmox-auto-installer/Cargo.toml | 14 +- proxmox-auto-installer/src/answer.rs | 27 +- .../src/bin/proxmox-auto-installer.rs | 13 +- proxmox-auto-installer/src/sysinfo.rs | 51 +- proxmox-auto-installer/src/udevinfo.rs | 8 +- proxmox-auto-installer/tests/parse-answer.rs | 13 +- proxmox-chroot/Cargo.toml | 7 +- proxmox-fetch-answer/Cargo.toml | 15 +- .../src/fetch_plugins/http.rs | 100 +-- proxmox-installer-common/Cargo.toml | 26 +- proxmox-installer-common/src/http.rs | 94 +++ proxmox-installer-common/src/lib.rs | 4 + proxmox-installer-common/src/options.rs | 5 + proxmox-installer-common/src/setup.rs | 2 +- proxmox-installer-common/src/sysinfo.rs | 52 ++ proxmox-installer-common/src/utils.rs | 2 + proxmox-post-hook/Cargo.toml | 18 + proxmox-post-hook/src/main.rs | 784 ++++++++++++++++++ proxmox-tui-installer/Cargo.toml | 8 +- unconfigured.sh | 5 +- 28 files changed, 1115 insertions(+), 217 deletions(-) create mode 100755 debian/scripts/elf-strip-unused-dependencies.sh create mode 100644 proxmox-installer-common/src/http.rs create mode 100644 proxmox-installer-common/src/sysinfo.rs create mode 100644 proxmox-post-hook/Cargo.toml create mode 100644 proxmox-post-hook/src/main.rs -- 2.45.1 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel