From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id DB4031FF2D6 for ; Mon, 15 Jul 2024 12:42:19 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EB5F8880; Mon, 15 Jul 2024 12:42:45 +0200 (CEST) Message-ID: <58501401-6e7e-4304-bede-4fe5cee03fb1@proxmox.com> Date: Mon, 15 Jul 2024 12:42:41 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta To: Proxmox VE development discussion , Christoph Heiss References: <20240710132756.1149508-1-c.heiss@proxmox.com> Content-Language: en-GB, de-AT From: Thomas Lamprecht Autocrypt: addr=t.lamprecht@proxmox.com; keydata= xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD 90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7 UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+ wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6 YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/ bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1 feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95 QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6 NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM 8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9 GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j /Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0 F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg 8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja 0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx In-Reply-To: <20240710132756.1149508-1-c.heiss@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL -0.052 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH installer 00/14] fix #5536: implement post-(auto-)installation notification mechanism X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" Am 10/07/2024 um 15:27 schrieb Christoph Heiss: > This implements a mechanism for post-installation "notifications" via a > POST request [0] when using the auto-installer. > > It's implemented as a separate, small utility to facilitate separation > of concerns and make the information gathering easier by having it > isolated in one place. > > Patches #1 through #10 are simply clean-ups, refactors, etc. that were > done along the way, which do not impact functionality in any way. > > Most interesting here will be patch #12, which adds the actual > implementation of the post-hook. (Bind-)mounting the installed host > system is done using the existing `proxmox-chroot` utility, and the HTTP > POST functionality can fortunately be re-used 1:1 from > `proxmox-fetch-answer`. > > I've also included an example of how the JSON body (pretty-printed for > readability) of such a post-installation request would look like below, > for reference. > > Tested this with both PVE and PBS ISOs, PMG did not (yet) have a > release with an auto-installation capable ISO. The only product-specific > code is the version detection in `proxmox-post-hook`, which - since it > works the same for PVE and PMG - be no obstacle. > > [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5536 > > { > "debian-version": "12.5", > "product-version": "pve-manager/8.2.2/9355359cd7afbae4", > "kernel-version": "proxmox-kernel-6.8.4-2-pve-signed", no hard feelings, but from a gut feeling I'd probably move this to a "version" object and potentially use a more reduced, easier to parse, value. Maybe also as an object so that both, a simple X.Y(.Z) release and a full string are given, as we changed (kernel) packages name in the past, and I could imagine that there will be a LTS and non LTS variant if we ever rework on where we base our kernel on, so this might change in the future too. While the simple X.Y.Z version will more likely stay as is. And I do not want to move the goal post here to far, but isn't some of this information potentially interesting to have sent to a metric server? At least with a low frequency (or just once on every boot) so that one has a somewhat recent externally saved set of information that can be used to identify machines more easily and be aware of some changes to correlate regressions or strange (load) metrics with. No need to do that now, but maybe something to keep in mind to allow easier reuse of this. IMO it's a big plus if we manage to keep information laid out the same way, or at list in a similar one, wherever it's included. And that doesn't have to mean that the whole struct has to be shared, maybe it just could be just a collection of types that stem from common crates outside the installer (at least in the long run, as said, no need to completely block this on scope extension now). > "boot-type": "bios", We call this "mode" in the product APIs [0], might potentially make sense to use the same schema here? Else I'd at least name this boot-mode and use the same keys. [0]: https://pve.proxmox.com/pve-docs/api-viewer/index.html#/nodes/{node}/status > "filesystem": "zfs (RAID1)", > "fqdn": "host.domain", > "machine-id": "f4bf9711783248b7aaffe3ccbca3e3dc", > "bootdisk": [ could be also interesting to get all disks and flag the ones used for booting with a boolean "bootdisk" flag. E.g. to make additional storage provisioning later on slightly more convenient. > { > "size": 8589934592, > "udev-properties": { > "DEVNAME": "/dev/vda", [..] > } > }, > { > "size": 8589934592, > "udev-properties": { > "DEVNAME": "/dev/vdb", [..] > } > } > ], > "management-nic": { > "mac": "de:ad:f0:0d:12:34", > "address": "10.0.0.10/24", > "udev-properties": { > "INTERFACE": "enp6s18", [..] > } > }, > "ssh-public-host-keys": { > "ecdsa": "ecdsa-sha2-nistp256 [..] root@host", > "ed25519": "ssh-ed25519 [..] root@host", > "rsa": "ssh-rsa [..] root@host", > } > } > > Christoph Heiss (14): chroot: print full anyhow message > tree-wide: fix some typos > tree-wide: collect hardcoded installer runtime directory strings into > constant > common: simplify filesystem type serializing & Display trait impl > common: setup: serialize `target_hd` as string explicitly > common: split out installer setup files loading functionality > debian: strip unused library dependencies > fetch-answer: move http-related code to gated module in > installer-common > tree-wide: convert some more crates to use workspace dependencies > auto-installer: tests: replace left/right with got/expected in output > auto-installer: answer: add `posthook` section > fix #5536: add post-hook utility for sending notifications after > auto-install > fix #5536: post-hook: add some unit tests > unconfigured.sh: run proxmox-post-hook after successful auto-install > > Cargo.toml | 11 + > Makefile | 8 +- > debian/control | 1 + > debian/install | 1 + > debian/rules | 9 + > proxmox-auto-install-assistant/Cargo.toml | 14 +- > proxmox-auto-installer/Cargo.toml | 15 +- > proxmox-auto-installer/src/answer.rs | 27 +- > .../src/bin/proxmox-auto-installer.rs | 15 +- > proxmox-auto-installer/src/sysinfo.rs | 10 +- > proxmox-auto-installer/src/utils.rs | 15 +- > proxmox-auto-installer/tests/parse-answer.rs | 42 +- > proxmox-chroot/Cargo.toml | 8 +- > proxmox-chroot/src/main.rs | 19 +- > proxmox-fetch-answer/Cargo.toml | 17 +- > .../src/fetch_plugins/http.rs | 100 +--- > .../src/fetch_plugins/partition.rs | 14 +- > proxmox-installer-common/Cargo.toml | 26 +- > proxmox-installer-common/src/http.rs | 94 ++++ > proxmox-installer-common/src/lib.rs | 5 + > proxmox-installer-common/src/options.rs | 109 ++-- > proxmox-installer-common/src/setup.rs | 108 +--- > proxmox-installer-common/src/utils.rs | 2 + > proxmox-post-hook/Cargo.toml | 19 + > proxmox-post-hook/src/main.rs | 498 ++++++++++++++++++ > proxmox-tui-installer/Cargo.toml | 8 +- > proxmox-tui-installer/src/setup.rs | 2 +- > unconfigured.sh | 7 +- > 28 files changed, 862 insertions(+), 342 deletions(-) > create mode 100644 proxmox-installer-common/src/http.rs > create mode 100644 proxmox-post-hook/Cargo.toml > create mode 100644 proxmox-post-hook/src/main.rs > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel