From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <a.lauterer@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 643E5C2689
for <pve-devel@lists.proxmox.com>; Tue, 23 Jan 2024 18:00:58 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 3C8A2AB5B
for <pve-devel@lists.proxmox.com>; Tue, 23 Jan 2024 18:00:58 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Tue, 23 Jan 2024 18:00:56 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id BC1F54926B
for <pve-devel@lists.proxmox.com>; Tue, 23 Jan 2024 18:00:56 +0100 (CET)
From: Aaron Lauterer <a.lauterer@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Tue, 23 Jan 2024 18:00:35 +0100
Message-Id: <20240123170053.490250-1-a.lauterer@proxmox.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.068 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
T_SCC_BODY_TEXT_LINE -0.01 -
Subject: [pve-devel] [PATCH v1 installer/docs 00/18] add
automated/unattended installation
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Tue, 23 Jan 2024 17:00:58 -0000
This patch series adds the possibility to do an automated / unattended
installation of Proxmox VE.
It assumes that the patch series to use JSON output on the
low-level-installer is already applied [1].
The overall idea is that we will have a dedicated ISO for the unattended
installation. It should be configured in such a way that it will start
the installation without any user interaction.
Though the integration in the installation environmend isn't part of
this patch series.
The information for the installer that is usually gathered interactively
from the user is provided via an `answer.toml` file.
The answer file allows to select disks and the network card via filters.
The installer also allows to run custom commands pre and post
installation. This should give users plenty of possibilities to either
further customize/prepare the installation or integrate it into a larger
automated installation setup.
For example, one could issue HTTP requests to signal the status and
progress of the installation.
The install environment needs to call the 'proxmox-fetch-answer' binary.
It tries to find the answer file and once found, will start the
'proxmox-auto-installer' binary and pass the contents to it via stdin.
The auto-installer then parses the answer file and determines what
parameters need to be passed to the low-level installer. For example,
which disks and NIC to use, network IP settings and so forth.
The current status reporting of the actual installation is kept rather
simple.
Both binaries log into the /tmp/ directory.
There is a third binary, the 'proxmox-installer-filter'. It is meant as
a pure utility for users to make it easier to see what properties they
can write filters against and to test the filters.
The fetch-answer binary is currently searching for a
partition/file-system labeled 'proxmoxinst' in lower or uppercase. It
can be located on an additioan USB flash drive, or maybe on the install
medium itself if it is possible to write to it.
We do have some ideas for additional steps to fetch an answer file. The
main one is that we could download the answer file from a URL. Ideally
we would send unique properties along with the request (MAC addresses,
serial numbers, ...) so that it is possible to have a script on the
receiving side that can then generate the answer file dynamically.
The big question is, where the URL comes from, for which we have also
some ideas:
* custom DHCP options
* kernel cmdline (might be an option with PXE boot)
* TXT DNS record in a predefined subdomain of the search domain received
via DHCP, basically a 'dig TXT proxmoxinst.{search domain}'.
* We should also make it possible to provide an SSL fingerprint in a
similar manner in case the listening server is not trusted out of the
box.
Other plans / ideas for the future:
* add option to define remote SSH access (password and,or public key).
This could make remote debugging in case of problems easier
Regarding the patch series itself:
The first patches are needed to move some code into the common crate and
make structs/functions already in the common crate accessible.
I did split up the individual parts of the auto installer into their own
patches as much as possible, and (hopefully) in the order they depend on
each other.
Areas that can be improved/extended:
* Testing possibility integrated in the Makefile
* Documentation: explain process, additional examples for answer.toml
[0] https://lists.proxmox.com/pipermail/pve-devel/2023-September/059020.html
[1] https://lists.proxmox.com/pipermail/pve-devel/2023-December/060961.html
installer: Aaron Lauterer (17):
tui: common: move InstallConfig struct to common crate
common: make InstallZfsOption members public
common: tui: use BTreeMap for predictable ordering
Makefile: fix handling of multiple usr_bin files
low-level: add dump-udev command
add auto-installer crate
auto-installer: add dependencies
auto-installer: add answer file definition
auto-installer: add struct to hold udev info
auto-installer: add utils
auto-installer: add simple logging
auto-installer: add tests for answer file parsing
auto-installer: add auto-installer binary
auto-installer: add fetch answer binary
auto-installer: use glob crate for pattern matching
auto-installer: utils: make get_udev_index functions public
auto-installer: add proxmox-installer-filter helper tool
docs: Aaron Lauterer (1):
installation: add unattended documentation
pve-installation.adoc | 267 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 267 insertions(+)
Cargo.toml | 1 +
Makefile | 9 +-
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 ++
proxmox-auto-installer/Cargo.toml | 19 +
proxmox-auto-installer/src/answer.rs | 148 ++++++
.../src/bin/proxmox-auto-installer.rs | 192 ++++++++
.../src/bin/proxmox-fetch-answer.rs | 73 +++
.../src/bin/proxmox-installer-filter.rs | 298 +++++++++++
.../src/fetch_plugins/mod.rs | 1 +
.../src/fetch_plugins/partition.rs | 102 ++++
proxmox-auto-installer/src/lib.rs | 5 +
proxmox-auto-installer/src/log.rs | 38 ++
proxmox-auto-installer/src/udevinfo.rs | 9 +
proxmox-auto-installer/src/utils.rs | 461 ++++++++++++++++++
proxmox-auto-installer/tests/parse-answer.rs | 102 ++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 ++
.../resources/parse_answer/disk_match.toml | 14 +
.../parse_answer/disk_match_all.json | 26 +
.../parse_answer/disk_match_all.toml | 16 +
.../parse_answer/disk_match_any.json | 33 ++
.../parse_answer/disk_match_any.toml | 16 +
.../tests/resources/parse_answer/minimal.json | 17 +
.../tests/resources/parse_answer/minimal.toml | 14 +
.../resources/parse_answer/nic_matching.json | 17 +
.../resources/parse_answer/nic_matching.toml | 19 +
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +
.../resources/parse_answer/specific_nic.toml | 19 +
.../tests/resources/parse_answer/zfs.json | 27 +
.../tests/resources/parse_answer/zfs.toml | 19 +
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
proxmox-installer-common/src/setup.rs | 100 +++-
proxmox-low-level-installer | 13 +
proxmox-tui-installer/src/options.rs | 4 +-
proxmox-tui-installer/src/setup.rs | 100 +---
.../src/views/install_progress.rs | 4 +-
40 files changed, 1915 insertions(+), 110 deletions(-)
create mode 100644 Proxmox/Sys/Udev.pm
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-installer-filter.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-auto-installer/src/lib.rs
create mode 100644 proxmox-auto-installer/src/log.rs
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
create mode 100644 proxmox-auto-installer/src/utils.rs
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
--
2.39.2