From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 21A39906C9 for ; Tue, 2 Apr 2024 16:43:35 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 02C247CCF for ; Tue, 2 Apr 2024 16:43:05 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 2 Apr 2024 16:43:03 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2C86644759 for ; Tue, 2 Apr 2024 16:43:03 +0200 (CEST) Date: Tue, 2 Apr 2024 16:43:00 +0200 From: Christoph Heiss To: Aaron Lauterer Cc: Proxmox VE development discussion Message-ID: References: <20240328135028.504520-1-a.lauterer@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240328135028.504520-1-a.lauterer@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL -0.000 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [mod.rs, partition.rs, answer.rs, lib.rs, setup.rs, parse-answer.rs, unconfigured.sh, udev.pm, main.rs, http.rs, post.rs, proxmox.com, udevinfo.rs, sysinfo.rs, utils.rs, log.rs, options.rs, proxmox-auto-installer.rs] Subject: Re: [pve-devel] [PATCH v3 00/30] add automated/unattended installation X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2024 14:43:35 -0000 Rebuild an (PVE) ISO to contain the new auto-installer with an appropriate GRUB entry setting `proxauto` on the kernel commandline. What I've tried: - Using a few different values for `global` options - Install on ext4, xfs, Btrfs RAID1 and ZFS RAID1 (with different values in multiple runs) - Using DHCP and static IP - Fetching answer from a HTTP source, getting the URL from DHCP - Trying out the `proxmox-autoinst-helper` tool for assembling udev rules and validating files. - Using the `post_command` to create some files in the newly installed system. I didn't play around all that extensively the udev filters. Some notes: - When using ext4 or xfs as filesystem, `disk_list` happily takes multiple disks (but really only installs on the first disk, of course). Should probably be another sanity check there. - As for the `proxmox-autoinst-helper answer` command, might `validate` or `validate-answer` be a better name? `answer` alone seems a bit confusing at first, as if e.g. the tool would give me an answer file or similar. - `{pre,post}_command` in the answer file should be spelled plural, as it is an array of commands after all. - While trying out different configurations, I wondered if for the network something like this would be better for static IPs: [network.manual] cidr = ".." dns = ".." [..] .. keeping the `network.use_dhcp` option as before. Would simplify some checks now and provide good future-proofing for any new options that might get added. Thereby basically modelling `proxmox_auto_installer::answer::NetworkSettings` enum nearly 1:1 to the TOML config. Nitpicking at this point, I know, but changing the answer format afterwards would be a quite a PITA :^) Overall very nice! and Tested-by: Christoph Heiss in any case. On Thu, Mar 28, 2024 at 02:49:58PM +0100, Aaron Lauterer wrote: > This patch series adds the possibility to do an automated / unattended > installation of Proxmox VE. > > The overall idea is that we will have a dedicated ISO for the unattended > installation. It should be configured in such a way that it will start > the installation without any user interaction. > Though the integration in the installation environmend isn't part of > this patch series. > > The information for the installer that is usually gathered interactively > from the user is provided via an `answer.toml` file. > > The answer file allows to select disks and the network card via filters. > > The installer also allows to run custom commands pre and post > installation. This should give users plenty of possibilities to either > further customize/prepare the installation or integrate it into a larger > automated installation setup. > For example, one could issue HTTP requests to signal the status and > progress of the installation. > > When the installer is called with 'proxauto' in the kernel cmdline, the > 'proxmox-fetch-answer' binary is called. It tries to find the answer > file and once found, will start the 'proxmox-auto-installer' binary and > pass the contents to it via stdin. > > The auto-installer then parses the answer file and determines what > parameters need to be passed to the low-level installer. For example, > which disks and NIC to use, network IP settings and so forth. > > The current status reporting of the actual installation is kept rather > simple. > > Both binaries log into the tmp directory. > > There is a third binary, the 'proxmox-autoinst-helper'. It provides a > few subcommands, from the help: > answer Validate if an answer file is formatted correctly > device-match Test which devices the given filter matches against > device-info Show device information that can be used for filters > identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file > > The fetch-answer binary is trying to get an answer file. It does so by > first searching for a partition/FS labeled `proxmoxinst`, or all upper > case, and an `answer.toml` in there. This could be provided by another > USB flash drive. > If that is not successful, the next step is to send an HTTP POST request > to a URL to get the TOML contents in return. A POST request was chosen > because we also send information to identify the host in JSON format. > > The question then is, where to get that URL from. Right now, there are > two options implemented. The first is looking for a custom DHCP option > and the second is querying for a TXT record in the `proxmoxinst` > subdomain of the search domain. > > It is possible to provide a SHA256 fingerprint of the SSL cert used by > the answer server. The safest option is to place a > `cert_fingerprint.txt` file in the same `proxmoxinst` partition as where > you alternatively would place the `answer.toml`. > If that is not found, then it can be provided by a second custom DHCP > option or placed as TXT record in the subdomain `proxmoxinst-fp`. > > This patch series now also separates the 3 binaries into their own > crate. The 'proxmox-fetch-answer' to keep the OpenSSL dependency as > localized as possible, and the 'proxmox-autoinst-helper' to make it easy > to compile just that binary. > > The new `proxmox-chroot` utility helps to prepare everything to chroot > into a fresh installation and clean it up once done. > This will be useful in the post commands when further customizing the > installation. > > > Other plans / ideas for the future: > > * add option to define remote SSH access (password and,or public key). > This could make remote debugging in case of problems easier > > > Regarding the patch series itself: > 01-03 are needed to move some code into the common crate and > make structs/functions already in the common crate accessible. > > I did split up the individual parts of the auto installer into their own > patches as much as possible, and (hopefully) in the order they depend on > each other. > > Patches after the `unconfigured` one (16), switch the pattern matching > to the glob crate, add the helper tool and the fetching via HTTP. > > Patch 25 factors our the binaries into their own crates. > > Patches 26-30 are for the 'proxmox-chroot' utility and preparations for > it to work. > > Areas that can be improved/extended: > * Testing possibility integrated in the Makefile > > I did test it with all 3 installers, PVE, PMG and PBS and it worked. > > WIP: writing documentation for it. Currently intended for the Wiki > as it can be used for all 3 products. > > Changes since v2: > * don't use 'dmidecode' but check in the source locations directly for > identifiers > * fixed makefile to get builds working every time > * reworked filesystem and raid level layout in the answer file > definition > * factor out binaries into their own crates > * add 'proxmox-chroot' helper tool > > Changes since v1: > * incorporated feedback (thx @cheiss) > * the proxmox-installer-filter tool got renamed and extended to the > proxmox-autoinst-helper > * added HTTP fetch functionality > * moved the fetch_plugins/utils.rs into its own subdirectory as it grew > quite a bit > * added builddeps > > [0] https://lists.proxmox.com/pipermail/pve-devel/2023-September/059020.html > > > > Aaron Lauterer (30): > tui: common: move InstallConfig struct to common crate > common: make InstallZfsOption members public > common: tui: use BTreeMap for predictable ordering > common: utils: add deserializer for CidrAddress > common: options: add Deserialize trait > low-level: add dump-udev command > add auto-installer crate > auto-installer: add dependencies > auto-installer: add answer file definition > auto-installer: add struct to hold udev info > auto-installer: add utils > auto-installer: add simple logging > auto-installer: add tests for answer file parsing > auto-installer: add auto-installer binary > auto-installer: add fetch answer binary > unconfigured: add proxauto as option to start auto installer > auto-installer: use glob crate for pattern matching > auto-installer: utils: make get_udev_index functions public > auto-installer: add proxmox-autoinst-helper tool > auto-installer: fetch: add gathering of system identifiers and > restructure code > auto-installer: helper: add subcommand to view indentifiers > auto-installer: fetch: add http post utility module > auto-installer: fetch: add http plugin to fetch answer > control: update build depends for auto installer > auto installer: factor out fetch-answer and autoinst-helper > low-level: write low level config to /tmp > common: add deserializer for FsType > common: skip target_hd when deserializing InstallConfig > common: add Display trait to ProxmoxProduct > add proxmox-chroot utility > > Cargo.toml | 4 + > Makefile | 19 +- > Proxmox/Makefile | 1 + > Proxmox/Sys/Udev.pm | 54 +++ > debian/control | 10 + > proxmox-auto-installer/Cargo.toml | 20 + > proxmox-auto-installer/src/answer.rs | 257 +++++++++++ > .../src/bin/proxmox-auto-installer.rs | 193 ++++++++ > proxmox-auto-installer/src/lib.rs | 5 + > proxmox-auto-installer/src/log.rs | 38 ++ > proxmox-auto-installer/src/sysinfo.rs | 81 ++++ > proxmox-auto-installer/src/udevinfo.rs | 9 + > proxmox-auto-installer/src/utils.rs | 430 ++++++++++++++++++ > proxmox-auto-installer/tests/parse-answer.rs | 102 +++++ > .../tests/resources/iso-info.json | 1 + > .../tests/resources/locales.json | 1 + > .../resources/parse_answer/disk_match.json | 29 ++ > .../resources/parse_answer/disk_match.toml | 17 + > .../parse_answer/disk_match_all.json | 26 ++ > .../parse_answer/disk_match_all.toml | 17 + > .../parse_answer/disk_match_any.json | 33 ++ > .../parse_answer/disk_match_any.toml | 17 + > .../tests/resources/parse_answer/minimal.json | 17 + > .../tests/resources/parse_answer/minimal.toml | 14 + > .../resources/parse_answer/nic_matching.json | 17 + > .../resources/parse_answer/nic_matching.toml | 19 + > .../tests/resources/parse_answer/readme | 4 + > .../resources/parse_answer/specific_nic.json | 17 + > .../resources/parse_answer/specific_nic.toml | 19 + > .../tests/resources/parse_answer/zfs.json | 27 ++ > .../tests/resources/parse_answer/zfs.toml | 20 + > .../tests/resources/run-env-info.json | 1 + > .../tests/resources/run-env-udev.json | 1 + > proxmox-autoinst-helper/Cargo.toml | 21 + > proxmox-autoinst-helper/src/main.rs | 337 ++++++++++++++ > proxmox-chroot/Cargo.toml | 16 + > proxmox-chroot/src/main.rs | 353 ++++++++++++++ > proxmox-fetch-answer/Cargo.toml | 22 + > .../src/fetch_plugins/http.rs | 191 ++++++++ > proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 + > .../src/fetch_plugins/partition.rs | 32 ++ > .../src/fetch_plugins/utils/mod.rs | 86 ++++ > .../src/fetch_plugins/utils/post.rs | 94 ++++ > proxmox-fetch-answer/src/main.rs | 76 ++++ > proxmox-installer-common/Cargo.toml | 1 + > proxmox-installer-common/src/options.rs | 21 +- > proxmox-installer-common/src/setup.rs | 154 ++++++- > proxmox-installer-common/src/utils.rs | 11 + > proxmox-low-level-installer | 14 + > proxmox-tui-installer/src/options.rs | 4 +- > proxmox-tui-installer/src/setup.rs | 100 +--- > .../src/views/install_progress.rs | 4 +- > unconfigured.sh | 17 + > 53 files changed, 2959 insertions(+), 118 deletions(-) > create mode 100644 Proxmox/Sys/Udev.pm > create mode 100644 proxmox-auto-installer/Cargo.toml > create mode 100644 proxmox-auto-installer/src/answer.rs > create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs > create mode 100644 proxmox-auto-installer/src/lib.rs > create mode 100644 proxmox-auto-installer/src/log.rs > create mode 100644 proxmox-auto-installer/src/sysinfo.rs > create mode 100644 proxmox-auto-installer/src/udevinfo.rs > create mode 100644 proxmox-auto-installer/src/utils.rs > create mode 100644 proxmox-auto-installer/tests/parse-answer.rs > create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json > create mode 100644 proxmox-auto-installer/tests/resources/locales.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json > create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml > create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json > create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json > create mode 100644 proxmox-autoinst-helper/Cargo.toml > create mode 100644 proxmox-autoinst-helper/src/main.rs > create mode 100644 proxmox-chroot/Cargo.toml > create mode 100644 proxmox-chroot/src/main.rs > create mode 100644 proxmox-fetch-answer/Cargo.toml > create mode 100644 proxmox-fetch-answer/src/fetch_plugins/http.rs > create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs > create mode 100644 proxmox-fetch-answer/src/fetch_plugins/partition.rs > create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs > create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/post.rs > create mode 100644 proxmox-fetch-answer/src/main.rs > > -- > 2.39.2 > > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > >