From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 1F6D29AA3 for ; Tue, 5 Sep 2023 15:29:06 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id AB4ED19F20 for ; Tue, 5 Sep 2023 15:28:35 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 5 Sep 2023 15:28:33 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9C7444226F for ; Tue, 5 Sep 2023 15:28:33 +0200 (CEST) From: Aaron Lauterer To: pve-devel@lists.proxmox.com Date: Tue, 5 Sep 2023 15:28:26 +0200 Message-Id: <20230905132832.3179097-1-a.lauterer@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.083 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [answer.rs, utils.rs, options.rs, mod.rs, setup.rs, udev.pm, udevinfo.rs, main.rs] Subject: [pve-devel] [RFC installer 0/6] add automated installation X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2023 13:29:06 -0000 This is the first iteration of making it possible to automatically run the installer. The main idea is to provide an answer file (TOML as of now) that provides the properties usually queried by the TUI or GUI installer. Additionally we want to be able to do some more fuzzy matching for the NIC and disks used by the installer. Therefore we have some basic globbing/wildcard support at the start and end of the search string. For now the UDEV device properties are used to for this matching. The format for the filters are "UDEV KEY -> search string". The answer file and auto installer have the additional option to run commands pre- and post-installation. More details can be found in the patch itself. The big question is how we actually get the answer file without creating a custom installer image per answer file. The most basic variant is to scan local storage for a partition/volume with an expected label, mount it and look for the expected file. For this I added a small shell script that does exactly this and then starts the auto installer. Another idea is to get an URL and query it * could come from a default subdomain that is queried within the DHCP provided search domain * We could get it from a DHCP option. How we extract that is something I don't know at this time. * From the kernel cmdline, if the installer is booted via PXE with customized kernel parameters * ... When running the http request, we could add identifying properties as parameters. Properties like MAC addresses and serial numbers, for example. This would make it possible to have a script querying an internal database and create the answer file on demand. This version definitely has some rough edges and probably a lot of things that could be done nicer, more idiomatic. There are quite a few nested for loops that could probably be done better/nicer as well. A lot of code has been reused from the TUI installer. The plan is to factor out common code into a new library crate. For now, the auto installer just prints everything to stdout. We could implement a simple GUI that shows a progress bar. pve-install: Aaron Lauterer (5): low level: sys: fetch udev properties add proxmox-auto-installer add answer file fetch script makefile: fix handling of multiple usr_bin files makefile: add auto installer Cargo.toml | 1 + Makefile | 8 +- Proxmox/Makefile | 1 + Proxmox/Sys/Udev.pm | 54 +++ proxmox-auto-installer/Cargo.toml | 13 + proxmox-auto-installer/answer.toml | 36 ++ .../resources/test/iso-info.json | 1 + .../resources/test/locales.json | 1 + .../test/parse_answer/disk_match.json | 28 ++ .../test/parse_answer/disk_match.toml | 14 + .../test/parse_answer/disk_match_all.json | 25 + .../test/parse_answer/disk_match_all.toml | 16 + .../test/parse_answer/disk_match_any.json | 32 ++ .../test/parse_answer/disk_match_any.toml | 16 + .../resources/test/parse_answer/minimal.json | 17 + .../resources/test/parse_answer/minimal.toml | 14 + .../test/parse_answer/nic_matching.json | 17 + .../test/parse_answer/nic_matching.toml | 19 + .../resources/test/parse_answer/readme | 4 + .../test/parse_answer/specific_nic.json | 17 + .../test/parse_answer/specific_nic.toml | 19 + .../resources/test/parse_answer/zfs.json | 26 + .../resources/test/parse_answer/zfs.toml | 19 + .../resources/test/run-env-info.json | 1 + .../resources/test/run-env-udev.json | 1 + proxmox-auto-installer/src/answer.rs | 144 ++++++ proxmox-auto-installer/src/main.rs | 412 ++++++++++++++++ proxmox-auto-installer/src/tui/mod.rs | 3 + proxmox-auto-installer/src/tui/options.rs | 302 ++++++++++++ proxmox-auto-installer/src/tui/setup.rs | 447 ++++++++++++++++++ proxmox-auto-installer/src/tui/utils.rs | 268 +++++++++++ proxmox-auto-installer/src/udevinfo.rs | 9 + proxmox-auto-installer/src/utils.rs | 325 +++++++++++++ proxmox-low-level-installer | 14 + start_autoinstall.sh | 50 ++ 35 files changed, 2372 insertions(+), 2 deletions(-) create mode 100644 Proxmox/Sys/Udev.pm create mode 100644 proxmox-auto-installer/Cargo.toml create mode 100644 proxmox-auto-installer/answer.toml create mode 100644 proxmox-auto-installer/resources/test/iso-info.json create mode 100644 proxmox-auto-installer/resources/test/locales.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match_all.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match_all.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match_any.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/disk_match_any.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/minimal.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/minimal.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/nic_matching.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/nic_matching.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/readme create mode 100644 proxmox-auto-installer/resources/test/parse_answer/specific_nic.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/specific_nic.toml create mode 100644 proxmox-auto-installer/resources/test/parse_answer/zfs.json create mode 100644 proxmox-auto-installer/resources/test/parse_answer/zfs.toml create mode 100644 proxmox-auto-installer/resources/test/run-env-info.json create mode 100644 proxmox-auto-installer/resources/test/run-env-udev.json create mode 100644 proxmox-auto-installer/src/answer.rs create mode 100644 proxmox-auto-installer/src/main.rs create mode 100644 proxmox-auto-installer/src/tui/mod.rs create mode 100644 proxmox-auto-installer/src/tui/options.rs create mode 100644 proxmox-auto-installer/src/tui/setup.rs create mode 100644 proxmox-auto-installer/src/tui/utils.rs create mode 100644 proxmox-auto-installer/src/udevinfo.rs create mode 100644 proxmox-auto-installer/src/utils.rs create mode 100755 start_autoinstall.sh pve-docs: Aaron Lauterer (1): installation: add unattended documentation pve-installation.adoc | 245 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 245 insertions(+) -- 2.39.2