* [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation
@ 2024-04-17 12:30 Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate Aaron Lauterer
` (35 more replies)
0 siblings, 36 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
patches until 31 got a [0,1]
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
changes since v5:
some small style fixups and using cert_fingerprint or TLS instead of
ssl_fingerprint and SSL in patches 32-36.
changes since v4:
Patches 32-36 finalize how to prepare an ISO for automated installation
and introduce a slight change in behavior, as it is now also possible to
include the needed parameters into the ISO directly:
* answer file itself
* URL to fetch it from
* SSL cert fingerprint
The 'proxmox-autoinst-helper' tool got a new subcommand to prepare an
ISO.
The cover letter iself:
The overall idea is to prepare an ISO for automated installation. A
prepare ISO will automatically boot into the installation.
The information for the installer that is usually gathered interactively
from the user is provided via an `answer.toml` file.
The answer file allows to select disks and the network card via filters.
The installer also allows to run custom commands pre and post
installation. This should give users plenty of possibilities to either
further customize/prepare the installation or integrate it into a larger
automated installation setup.
For example, one could issue HTTP requests to signal the status and
progress of the installation.
When the installer is called with 'proxauto' in the kernel cmdline, the
'proxmox-fetch-answer' binary is called. It tries to find the answer
file and once found, will start the 'proxmox-auto-installer' binary and
pass the contents to it via stdin.
The auto-installer then parses the answer file and determines what
parameters need to be passed to the low-level installer. For example,
which disks and NIC to use, network IP settings and so forth.
The current status reporting of the actual installation is kept rather
simple.
Both binaries log into the tmp directory.
There is a third binary, the 'proxmox-autoinst-helper'. It provides a
few subcommands, from the help:
prepare-iso Prepare an ISO for automated installation.
validate-answer Validate if an answer file is formatted correctly
device-match Test which devices the given filter matches against
device-info Show device information that can be used for filters
identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file
help Print this message or the help of the given subcommand(s)
The fetch-answer binary is trying to get an answer file. If included in
the ISO, it will use that one. If no answer file is included, it first
searches for a partition/FS labeled `proxmoxinst`, or all upper case,
and an `answer.toml` in there. This could be provided by another USB
flash drive.
If that is not successful, the next step is to send an HTTP POST request
to a URL to get the TOML contents in return. A POST request was chosen
because we also send information to identify the host in JSON format.
The question then is, where to get that URL from. Right now, there are
three options implemented. The first is to hardcode it in the prepared
iso. The second is to look for a custom DHCP option
and the third is to query for a TXT record in the `proxmoxinst`
subdomain of the search domain.
It is possible to provide a SHA256 fingerprint of the SSL cert used by
the answer server. The safest option is to include it in the ISO itself.
If that is not found, then it can be provided by a second custom DHCP
option or placed as TXT record in the subdomain `proxmoxinst-fp`.
This patch series now also separates the 3 binaries into their own
crate. The 'proxmox-fetch-answer' to keep the OpenSSL dependency as
localized as possible, and the 'proxmox-autoinst-helper' to make it easy
to compile just that binary.
The new `proxmox-chroot` utility helps to prepare everything to chroot
into a fresh installation and clean it up once done.
This will be useful in the post commands when further customizing the
installation.
Other plans / ideas for the future:
* add option to define remote SSH access (password and,or public key).
This could make remote debugging in case of problems easier
* make it possible to flash prepared ISO directly to USB flash drives
Regarding the patch series itself:
01-03 are needed to move some code into the common crate and
make structs/functions already in the common crate accessible.
I did split up the individual parts of the auto installer into their own
patches as much as possible, and (hopefully) in the order they depend on
each other.
Patches after the `unconfigured` one (16), switch the pattern matching
to the glob crate, add the helper tool and the fetching via HTTP.
Patch 26 factors our the binaries into their own crates.
Patches 27-30 are for the 'proxmox-chroot' utility and preparations for
it to work.
Patch 31 makes sure that the answer file can only contain known keys.
Patches 32 - 36 finalize the ISO preparation, add the subcommand for it
to the autoinst-helper tool and adapt the fetch-answer binary to also
handle the new options to include the infos in the ISO itself.
Areas that can be improved/extended:
* Testing possibility integrated in the Makefile
* make build target for statically compiled proxmox-autoinst-helper
I did test it with all 3 installers, PVE, PMG and PBS and it worked.
With PVE I tested all variants on how to inject infos into the prepared
ISO.
WIP: Documentation. A first draft is available in the inernal wiki, as
we will most likely keep it in wiki format since it applies for all 3
products, if we provide ISOs for it.
[0] https://lists.proxmox.com/pipermail/pve-devel/2024-April/062634.html
[1] https://lists.proxmox.com/pipermail/pve-devel/2024-April/062690.html
Aaron Lauterer (36):
tui: common: move InstallConfig struct to common crate
common: make InstallZfsOption members public
common: tui: use BTreeMap for predictable ordering
common: utils: add deserializer for CidrAddress
common: options: add Deserialize trait
low-level: add dump-udev command
add auto-installer crate
auto-installer: add dependencies
auto-installer: add answer file definition
auto-installer: add struct to hold udev info
auto-installer: add utils
auto-installer: add simple logging
auto-installer: add tests for answer file parsing
auto-installer: add auto-installer binary
auto-installer: add fetch answer binary
unconfigured: add proxauto as option to start auto installer
auto-installer: use glob crate for pattern matching
auto-installer: utils: make get_udev_index functions public
auto-installer: add proxmox-autoinst-helper tool
common: add Display trait to ProxmoxProduct
auto-installer: fetch: add gathering of system identifiers and
restructure code
auto-installer: helper: add subcommand to view indentifiers
auto-installer: fetch: add http post utility module
auto-installer: fetch: add http plugin to fetch answer
control: update build depends for auto installer
auto installer: factor out fetch-answer and autoinst-helper
low-level: write low level config to /tmp
common: add deserializer for FsType
common: skip target_hd when deserializing InstallConfig
add proxmox-chroot utility
auto-installer: answer: deny unknown fields
fetch-answer: move get_answer_file to utils
auto-installer: utils: define ISO specified settings
fetch-answer: use ISO specified configurations
fetch-answer: dpcp: improve logging of steps taken
autoinst-helper: add prepare-iso subcommand
Cargo.toml | 4 +
Makefile | 19 +-
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 ++
debian/control | 10 +
proxmox-auto-installer/Cargo.toml | 20 +
proxmox-auto-installer/src/answer.rs | 255 ++++++++
.../src/bin/proxmox-auto-installer.rs | 195 ++++++
proxmox-auto-installer/src/lib.rs | 5 +
proxmox-auto-installer/src/log.rs | 38 ++
proxmox-auto-installer/src/sysinfo.rs | 81 +++
proxmox-auto-installer/src/udevinfo.rs | 9 +
proxmox-auto-installer/src/utils.rs | 455 ++++++++++++++
proxmox-auto-installer/tests/parse-answer.rs | 106 ++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 +
.../resources/parse_answer/disk_match.toml | 17 +
.../parse_answer/disk_match_all.json | 26 +
.../parse_answer/disk_match_all.toml | 17 +
.../parse_answer/disk_match_any.json | 33 +
.../parse_answer/disk_match_any.toml | 17 +
.../tests/resources/parse_answer/minimal.json | 17 +
.../tests/resources/parse_answer/minimal.toml | 14 +
.../resources/parse_answer/nic_matching.json | 17 +
.../resources/parse_answer/nic_matching.toml | 19 +
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +
.../resources/parse_answer/specific_nic.toml | 19 +
.../tests/resources/parse_answer/zfs.json | 27 +
.../tests/resources/parse_answer/zfs.toml | 20 +
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
proxmox-autoinst-helper/Cargo.toml | 22 +
proxmox-autoinst-helper/src/main.rs | 562 ++++++++++++++++++
proxmox-chroot/Cargo.toml | 16 +
proxmox-chroot/src/main.rs | 356 +++++++++++
proxmox-fetch-answer/Cargo.toml | 23 +
.../src/fetch_plugins/http.rs | 182 ++++++
proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 +
.../src/fetch_plugins/partition.rs | 21 +
.../src/fetch_plugins/utils/mod.rs | 96 +++
.../src/fetch_plugins/utils/post.rs | 94 +++
proxmox-fetch-answer/src/main.rs | 116 ++++
proxmox-installer-common/Cargo.toml | 1 +
proxmox-installer-common/src/options.rs | 21 +-
proxmox-installer-common/src/setup.rs | 141 ++++-
proxmox-installer-common/src/utils.rs | 11 +
proxmox-low-level-installer | 14 +
proxmox-tui-installer/src/options.rs | 4 +-
proxmox-tui-installer/src/setup.rs | 100 +---
.../src/views/install_progress.rs | 4 +-
unconfigured.sh | 17 +
53 files changed, 3235 insertions(+), 118 deletions(-)
create mode 100644 Proxmox/Sys/Udev.pm
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
create mode 100644 proxmox-auto-installer/src/lib.rs
create mode 100644 proxmox-auto-installer/src/log.rs
create mode 100644 proxmox-auto-installer/src/sysinfo.rs
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
create mode 100644 proxmox-auto-installer/src/utils.rs
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
create mode 100644 proxmox-autoinst-helper/Cargo.toml
create mode 100644 proxmox-autoinst-helper/src/main.rs
create mode 100644 proxmox-chroot/Cargo.toml
create mode 100644 proxmox-chroot/src/main.rs
create mode 100644 proxmox-fetch-answer/Cargo.toml
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/http.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
create mode 100644 proxmox-fetch-answer/src/main.rs
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 02/36] common: make InstallZfsOption members public Aaron Lauterer
` (34 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
It describes the data structure expected by the low-level-installer.
We do this so we can use it in more than the TUI installer, for example
the planned auto installer.
Make the members public so we can easily implement a custom From method
for each dependent crate.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 86 +++++++++++++++-
proxmox-tui-installer/src/setup.rs | 98 +------------------
.../src/views/install_progress.rs | 4 +-
3 files changed, 90 insertions(+), 98 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 472e1f2..03beb77 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -12,7 +12,10 @@ use std::{
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
use crate::{
- options::{Disk, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption},
+ options::{
+ BtrfsRaidLevel, Disk, FsType, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption,
+ ZfsRaidLevel,
+ },
utils::CidrAddress,
};
@@ -387,3 +390,84 @@ pub fn spawn_low_level_installer(test_mode: bool) -> io::Result<process::Child>
.stdout(Stdio::piped())
.spawn()
}
+
+/// See Proxmox::Install::Config
+#[derive(Serialize)]
+pub struct InstallConfig {
+ pub autoreboot: usize,
+
+ #[serde(serialize_with = "serialize_fstype")]
+ pub filesys: FsType,
+ pub hdsize: f64,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub swapsize: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub maxroot: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub minfree: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub maxvz: Option<f64>,
+
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub zfs_opts: Option<InstallZfsOption>,
+
+ #[serde(
+ serialize_with = "serialize_disk_opt",
+ skip_serializing_if = "Option::is_none"
+ )]
+ pub target_hd: Option<Disk>,
+ #[serde(skip_serializing_if = "HashMap::is_empty")]
+ pub disk_selection: HashMap<String, String>,
+
+ pub country: String,
+ pub timezone: String,
+ pub keymap: String,
+
+ pub password: String,
+ pub mailto: String,
+
+ pub mngmt_nic: String,
+
+ pub hostname: String,
+ pub domain: String,
+ #[serde(serialize_with = "serialize_as_display")]
+ pub cidr: CidrAddress,
+ pub gateway: IpAddr,
+ pub dns: IpAddr,
+}
+
+fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error>
+where
+ S: Serializer,
+{
+ if let Some(disk) = value {
+ serializer.serialize_str(&disk.path)
+ } else {
+ serializer.serialize_none()
+ }
+}
+
+fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error>
+where
+ S: Serializer,
+{
+ use FsType::*;
+ let value = match value {
+ // proxinstall::$fssetup
+ Ext4 => "ext4",
+ Xfs => "xfs",
+ // proxinstall::get_zfs_raid_setup()
+ Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)",
+ Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)",
+ Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)",
+ Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)",
+ Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)",
+ Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)",
+ // proxinstall::get_btrfs_raid_setup()
+ Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)",
+ Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)",
+ Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)",
+ };
+
+ serializer.collect_str(value)
+}
diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs
index 79421d7..e816c12 100644
--- a/proxmox-tui-installer/src/setup.rs
+++ b/proxmox-tui-installer/src/setup.rs
@@ -1,59 +1,11 @@
-use std::{collections::HashMap, fmt, net::IpAddr};
-
-use serde::{Serialize, Serializer};
+use std::collections::HashMap;
use crate::options::InstallerOptions;
use proxmox_installer_common::{
- options::{AdvancedBootdiskOptions, BtrfsRaidLevel, Disk, FsType, ZfsRaidLevel},
- setup::InstallZfsOption,
- utils::CidrAddress,
+ options::AdvancedBootdiskOptions,
+ setup::InstallConfig,
};
-/// See Proxmox::Install::Config
-#[derive(Serialize)]
-pub struct InstallConfig {
- autoreboot: usize,
-
- #[serde(serialize_with = "serialize_fstype")]
- filesys: FsType,
- hdsize: f64,
- #[serde(skip_serializing_if = "Option::is_none")]
- swapsize: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- maxroot: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- minfree: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- maxvz: Option<f64>,
-
- #[serde(skip_serializing_if = "Option::is_none")]
- zfs_opts: Option<InstallZfsOption>,
-
- #[serde(
- serialize_with = "serialize_disk_opt",
- skip_serializing_if = "Option::is_none"
- )]
- target_hd: Option<Disk>,
- #[serde(skip_serializing_if = "HashMap::is_empty")]
- disk_selection: HashMap<String, String>,
-
- country: String,
- timezone: String,
- keymap: String,
-
- password: String,
- mailto: String,
-
- mngmt_nic: String,
-
- hostname: String,
- domain: String,
- #[serde(serialize_with = "serialize_as_display")]
- cidr: CidrAddress,
- gateway: IpAddr,
- dns: IpAddr,
-}
-
impl From<InstallerOptions> for InstallConfig {
fn from(options: InstallerOptions) -> Self {
let mut config = Self {
@@ -121,47 +73,3 @@ impl From<InstallerOptions> for InstallConfig {
config
}
}
-
-fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
-{
- if let Some(disk) = value {
- serializer.serialize_str(&disk.path)
- } else {
- serializer.serialize_none()
- }
-}
-
-fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
-{
- use FsType::*;
- let value = match value {
- // proxinstall::$fssetup
- Ext4 => "ext4",
- Xfs => "xfs",
- // proxinstall::get_zfs_raid_setup()
- Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)",
- Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)",
- Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)",
- Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)",
- Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)",
- Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)",
- // proxinstall::get_btrfs_raid_setup()
- Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)",
- Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)",
- Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)",
- };
-
- serializer.collect_str(value)
-}
-
-fn serialize_as_display<S, T>(value: &T, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
- T: fmt::Display,
-{
- serializer.collect_str(value)
-}
diff --git a/proxmox-tui-installer/src/views/install_progress.rs b/proxmox-tui-installer/src/views/install_progress.rs
index c2c9ddf..56356ad 100644
--- a/proxmox-tui-installer/src/views/install_progress.rs
+++ b/proxmox-tui-installer/src/views/install_progress.rs
@@ -13,8 +13,8 @@ use std::{
time::Duration,
};
-use crate::{abort_install_button, prompt_dialog, setup::InstallConfig, InstallerState};
-use proxmox_installer_common::setup::spawn_low_level_installer;
+use crate::{abort_install_button, prompt_dialog, InstallerState};
+use proxmox_installer_common::setup::{InstallConfig, spawn_low_level_installer};
pub struct InstallProgressView {
view: PaddedView<LinearLayout>,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 02/36] common: make InstallZfsOption members public
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 03/36] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
` (33 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
as they will be used directly by the auto installer
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 03beb77..1bc4cb7 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -147,13 +147,13 @@ pub fn installer_setup(in_test_mode: bool) -> Result<(SetupInfo, LocaleInfo, Run
#[derive(Serialize)]
pub struct InstallZfsOption {
- ashift: usize,
+ pub ashift: usize,
#[serde(serialize_with = "serialize_as_display")]
- compress: ZfsCompressOption,
+ pub compress: ZfsCompressOption,
#[serde(serialize_with = "serialize_as_display")]
- checksum: ZfsChecksumOption,
- copies: usize,
- arc_max: usize,
+ pub checksum: ZfsChecksumOption,
+ pub copies: usize,
+ pub arc_max: usize,
}
impl From<ZfsBootdiskOptions> for InstallZfsOption {
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 03/36] common: tui: use BTreeMap for predictable ordering
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 02/36] common: make InstallZfsOption members public Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 04/36] common: utils: add deserializer for CidrAddress Aaron Lauterer
` (32 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
necessary for the disk selection and network interfaces maps to have
tests with results that can be compared without much additional effort.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 8 ++++----
proxmox-tui-installer/src/options.rs | 4 ++--
proxmox-tui-installer/src/setup.rs | 4 ++--
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 1bc4cb7..8432a2c 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -1,6 +1,6 @@
use std::{
cmp,
- collections::HashMap,
+ collections::{BTreeMap, HashMap},
fmt,
fs::File,
io::{self, BufReader},
@@ -297,7 +297,7 @@ pub struct NetworkInfo {
/// Maps devices to their configuration, if it has a usable configuration.
/// (Contains no entries for devices with only link-local addresses.)
#[serde(default)]
- pub interfaces: HashMap<String, Interface>,
+ pub interfaces: BTreeMap<String, Interface>,
/// The hostname of this machine, if set by the DHCP server.
pub hostname: Option<String>,
@@ -416,8 +416,8 @@ pub struct InstallConfig {
skip_serializing_if = "Option::is_none"
)]
pub target_hd: Option<Disk>,
- #[serde(skip_serializing_if = "HashMap::is_empty")]
- pub disk_selection: HashMap<String, String>,
+ #[serde(skip_serializing_if = "BTreeMap::is_empty")]
+ pub disk_selection: BTreeMap<String, String>,
pub country: String,
pub timezone: String,
diff --git a/proxmox-tui-installer/src/options.rs b/proxmox-tui-installer/src/options.rs
index 094a430..73fbf2a 100644
--- a/proxmox-tui-installer/src/options.rs
+++ b/proxmox-tui-installer/src/options.rs
@@ -76,7 +76,7 @@ mod tests {
utils::{CidrAddress, Fqdn},
};
use std::net::{IpAddr, Ipv4Addr};
- use std::{collections::HashMap, path::PathBuf};
+ use std::{collections::BTreeMap, path::PathBuf};
fn dummy_setup_info() -> SetupInfo {
SetupInfo {
@@ -99,7 +99,7 @@ mod tests {
fn network_options_from_setup_network_info() {
let setup = dummy_setup_info();
- let mut interfaces = HashMap::new();
+ let mut interfaces = BTreeMap::new();
interfaces.insert(
"eth0".to_owned(),
Interface {
diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs
index e816c12..248f86e 100644
--- a/proxmox-tui-installer/src/setup.rs
+++ b/proxmox-tui-installer/src/setup.rs
@@ -1,4 +1,4 @@
-use std::collections::HashMap;
+use std::collections::BTreeMap;
use crate::options::InstallerOptions;
use proxmox_installer_common::{
@@ -19,7 +19,7 @@ impl From<InstallerOptions> for InstallConfig {
maxvz: None,
zfs_opts: None,
target_hd: None,
- disk_selection: HashMap::new(),
+ disk_selection: BTreeMap::new(),
country: options.timezone.country,
timezone: options.timezone.timezone,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 04/36] common: utils: add deserializer for CidrAddress
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (2 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 03/36] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 05/36] common: options: add Deserialize trait Aaron Lauterer
` (31 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
so it can be deserialized from a string
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/utils.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/proxmox-installer-common/src/utils.rs b/proxmox-installer-common/src/utils.rs
index 36b1d53..f6521eb 100644
--- a/proxmox-installer-common/src/utils.rs
+++ b/proxmox-installer-common/src/utils.rs
@@ -103,6 +103,17 @@ impl fmt::Display for CidrAddress {
}
}
+impl<'de> Deserialize<'de> for CidrAddress {
+ fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
+ where
+ D: serde::Deserializer<'de>,
+ {
+ let s: String = Deserialize::deserialize(deserializer)?;
+ s.parse()
+ .map_err(|_| serde::de::Error::custom("invalid CIDR"))
+ }
+}
+
fn mask_limit(addr: &IpAddr) -> usize {
if addr.is_ipv4() {
32
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 05/36] common: options: add Deserialize trait
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (3 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 04/36] common: utils: add deserializer for CidrAddress Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 06/36] low-level: add dump-udev command Aaron Lauterer
` (30 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
For the Enums that will be used to deserialize an answer file.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/options.rs | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/proxmox-installer-common/src/options.rs b/proxmox-installer-common/src/options.rs
index 1aa8f65..1efac66 100644
--- a/proxmox-installer-common/src/options.rs
+++ b/proxmox-installer-common/src/options.rs
@@ -1,12 +1,14 @@
use std::net::{IpAddr, Ipv4Addr};
use std::{cmp, fmt};
+use serde::Deserialize;
use crate::setup::{
LocaleInfo, NetworkInfo, ProductConfig, ProxmoxProduct, RuntimeInfo, SetupInfo,
};
use crate::utils::{CidrAddress, Fqdn};
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum BtrfsRaidLevel {
Raid0,
Raid1,
@@ -24,7 +26,8 @@ impl fmt::Display for BtrfsRaidLevel {
}
}
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum ZfsRaidLevel {
Raid0,
Raid1,
@@ -112,7 +115,8 @@ impl BtrfsBootdiskOptions {
}
}
-#[derive(Copy, Clone, Debug, Default, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Default, Deserialize, Eq, PartialEq)]
+#[serde(rename_all(deserialize = "lowercase"))]
pub enum ZfsCompressOption {
#[default]
On,
@@ -141,7 +145,8 @@ pub const ZFS_COMPRESS_OPTIONS: &[ZfsCompressOption] = {
&[On, Off, Lzjb, Lz4, Zle, Gzip, Zstd]
};
-#[derive(Copy, Clone, Debug, Default, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Default, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "kebab-case")]
pub enum ZfsChecksumOption {
#[default]
On,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 06/36] low-level: add dump-udev command
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (4 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 05/36] common: options: add Deserialize trait Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 07/36] add auto-installer crate Aaron Lauterer
` (29 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Fetches UDEV device properties prepended with 'E:' for NICs and disks.
The result is stored in its own JSON file.
This information is needed to filter for specific devices. Mainly for
the auto-installer for now.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 +++++++++++++++++++++++++++++++++++++
proxmox-low-level-installer | 13 +++++++++
3 files changed, 68 insertions(+)
create mode 100644 Proxmox/Sys/Udev.pm
diff --git a/Proxmox/Makefile b/Proxmox/Makefile
index d49da80..9561d9b 100644
--- a/Proxmox/Makefile
+++ b/Proxmox/Makefile
@@ -16,6 +16,7 @@ PERL_MODULES=\
Sys/Command.pm \
Sys/File.pm \
Sys/Net.pm \
+ Sys/Udev.pm \
UI.pm \
UI/Base.pm \
UI/Gtk3.pm \
diff --git a/Proxmox/Sys/Udev.pm b/Proxmox/Sys/Udev.pm
new file mode 100644
index 0000000..69d674f
--- /dev/null
+++ b/Proxmox/Sys/Udev.pm
@@ -0,0 +1,54 @@
+package Proxmox::Sys::Udev;
+
+use strict;
+use warnings;
+
+use base qw(Exporter);
+our @EXPORT_OK = qw(disk_details);
+
+my $udev_regex = '^E: ([A-Z_]*)=(.*)$';
+
+my sub fetch_udevadm_info {
+ my ($path) = @_;
+
+ my $info = `udevadm info --path $path --query all`;
+ if (!$info) {
+ warn "no details found for device '${path}'\n";
+ next;
+ }
+ my $details = {};
+ for my $line (split('\n', $info)) {
+ if ($line =~ m/$udev_regex/) {
+ $details->{$1} = $2;
+ }
+ }
+ return $details;
+}
+
+# return hash of E: properties returned by udevadm
+sub disk_details {
+ my $result = {};
+ for my $data (@{Proxmox::Sys::Block::get_cached_disks()}) {
+ my $index = @$data[0];
+ my $bd = @$data[5];
+ $result->{$index} = fetch_udevadm_info($bd);
+ }
+ return $result;
+}
+
+
+sub nic_details {
+ my $nic_path = "/sys/class/net";
+ my $result = {};
+
+ my $nics = Proxmox::Sys::Net::get_ip_config()->{ifaces};
+
+ for my $index (keys %$nics) {
+ my $name = $nics->{$index}->{name};
+ my $nic = "${nic_path}/${name}";
+ $result->{$name} = fetch_udevadm_info($nic);
+ }
+ return $result;
+}
+
+1;
diff --git a/proxmox-low-level-installer b/proxmox-low-level-installer
index 2848295..54f689a 100755
--- a/proxmox-low-level-installer
+++ b/proxmox-low-level-installer
@@ -21,6 +21,7 @@ use Time::HiRes qw(usleep);
use Proxmox::Install::ISOEnv;
use Proxmox::Install::RunEnv;
+use Proxmox::Sys::Udev;
use Proxmox::Sys::File qw(file_write_all);
@@ -31,6 +32,7 @@ use Proxmox::UI;
my $commands = {
'dump-env' => 'Dump the current ISO and Hardware environment to base the installer UI on.',
+ 'dump-udev' => 'Dump disk and network device info. Used for the auto installation.',
'start-session' => 'Start an installation session, with command and result transmitted via stdin/out',
'start-session-test' => 'Start an installation TEST session, with command and result transmitted via stdin/out',
'help' => 'Output this usage help.',
@@ -115,6 +117,17 @@ if ($cmd eq 'dump-env') {
my $run_env = Proxmox::Install::RunEnv::query_installation_environment();
my $run_env_serialized = to_json($run_env, {canonical => 1, utf8 => 1}) ."\n";
file_write_all($run_env_file, $run_env_serialized);
+} elsif ($cmd eq 'dump-udev') {
+ my $out_dir = $env->{locations}->{run};
+ make_path($out_dir);
+ die "failed to create output directory '$out_dir'\n" if !-d $out_dir;
+
+ my $output = {};
+ $output->{disks} = Proxmox::Sys::Udev::disk_details();
+ $output->{nics} = Proxmox::Sys::Udev::nic_details();
+
+ my $output_serialized = to_json($output, {canonical => 1, utf8 => 1}) ."\n";
+ file_write_all("$out_dir/run-env-udev.json", $output_serialized);
} elsif ($cmd eq 'start-session') {
Proxmox::UI::init_stdio({}, $env);
read_and_merge_config();
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 07/36] add auto-installer crate
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (5 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 06/36] low-level: add dump-udev command Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 08/36] auto-installer: add dependencies Aaron Lauterer
` (28 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 1 +
Makefile | 1 +
proxmox-auto-installer/Cargo.toml | 10 ++++++++++
proxmox-auto-installer/src/lib.rs | 0
4 files changed, 12 insertions(+)
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index c1bd578..7017ac5 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,6 @@
[workspace]
members = [
+ "proxmox-auto-installer",
"proxmox-installer-common",
"proxmox-tui-installer",
]
diff --git a/Makefile b/Makefile
index af33e90..4f140ca 100644
--- a/Makefile
+++ b/Makefile
@@ -47,6 +47,7 @@ $(BUILDDIR):
interfaces \
proxinstall \
proxmox-low-level-installer \
+ proxmox-auto-installer/ \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
new file mode 100644
index 0000000..75cfb2c
--- /dev/null
+++ b/proxmox-auto-installer/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "proxmox-auto-installer"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+[dependencies]
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
new file mode 100644
index 0000000..e69de29
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 08/36] auto-installer: add dependencies
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (6 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 07/36] add auto-installer crate Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 09/36] auto-installer: add answer file definition Aaron Lauterer
` (27 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 75cfb2c..67218dd 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -8,3 +8,7 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+proxmox-installer-common = { path = "../proxmox-installer-common" }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+toml = "0.7"
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 09/36] auto-installer: add answer file definition
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (7 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 08/36] auto-installer: add dependencies Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 10/36] auto-installer: add struct to hold udev info Aaron Lauterer
` (26 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 1 +
proxmox-auto-installer/src/answer.rs | 248 +++++++++++++++++++++++++++
proxmox-auto-installer/src/lib.rs | 1 +
3 files changed, 250 insertions(+)
create mode 100644 proxmox-auto-installer/src/answer.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 67218dd..80de4fa 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -12,3 +12,4 @@ proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.7"
+enum-iterator = "0.6.0"
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
new file mode 100644
index 0000000..0add95e
--- /dev/null
+++ b/proxmox-auto-installer/src/answer.rs
@@ -0,0 +1,248 @@
+use proxmox_installer_common::{
+ options::{BtrfsRaidLevel, FsType, ZfsChecksumOption, ZfsCompressOption, ZfsRaidLevel},
+ utils::{CidrAddress, Fqdn},
+};
+use serde::{Deserialize, Serialize};
+use std::{collections::BTreeMap, net::IpAddr};
+
+/// BTreeMap is used to store filters as the order of the filters will be stable, compared to
+/// storing them in a HashMap
+
+#[derive(Clone, Deserialize, Debug)]
+#[serde(rename_all = "kebab-case")]
+pub struct Answer {
+ pub global: Global,
+ pub network: Network,
+ #[serde(rename = "disk-setup")]
+ pub disks: Disks,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+pub struct Global {
+ pub country: String,
+ pub fqdn: Fqdn,
+ pub keyboard: String,
+ pub mailto: String,
+ pub timezone: String,
+ pub password: String,
+ pub pre_commands: Option<Vec<String>>,
+ pub post_commands: Option<Vec<String>>,
+ #[serde(default)]
+ pub reboot_on_error: bool,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+struct NetworkInAnswer {
+ #[serde(default)]
+ pub use_dhcp: bool,
+ pub cidr: Option<CidrAddress>,
+ pub dns: Option<IpAddr>,
+ pub gateway: Option<IpAddr>,
+ pub filter: Option<BTreeMap<String, String>>,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+#[serde(try_from = "NetworkInAnswer")]
+pub struct Network {
+ pub network_settings: NetworkSettings,
+}
+
+impl TryFrom<NetworkInAnswer> for Network {
+ type Error = &'static str;
+
+ fn try_from(source: NetworkInAnswer) -> Result<Self, Self::Error> {
+ if !source.use_dhcp {
+ if source.cidr.is_none() {
+ return Err("Field 'cidr' must be set.");
+ }
+ if source.dns.is_none() {
+ return Err("Field 'dns' must be set.");
+ }
+ if source.gateway.is_none() {
+ return Err("Field 'gateway' must be set.");
+ }
+ if source.filter.is_none() {
+ return Err("Field 'filter' must be set.");
+ }
+
+ Ok(Network {
+ network_settings: NetworkSettings::Manual(NetworkManual {
+ cidr: source.cidr.unwrap(),
+ dns: source.dns.unwrap(),
+ gateway: source.gateway.unwrap(),
+ filter: source.filter.unwrap(),
+ }),
+ })
+ } else {
+ Ok(Network {
+ network_settings: NetworkSettings::Dhcp(true),
+ })
+ }
+ }
+}
+
+#[derive(Clone, Debug)]
+pub enum NetworkSettings {
+ Dhcp(bool),
+ Manual(NetworkManual),
+}
+
+#[derive(Clone, Debug)]
+pub struct NetworkManual {
+ pub cidr: CidrAddress,
+ pub dns: IpAddr,
+ pub gateway: IpAddr,
+ pub filter: BTreeMap<String, String>,
+}
+
+#[derive(Clone, Debug, Deserialize)]
+pub struct DiskSetup {
+ pub filesystem: Filesystem,
+ #[serde(default)]
+ pub disk_list: Vec<String>,
+ pub filter: Option<BTreeMap<String, String>>,
+ pub filter_match: Option<FilterMatch>,
+ pub zfs: Option<ZfsOptions>,
+ pub lvm: Option<LvmOptions>,
+ pub btrfs: Option<BtrfsOptions>,
+}
+
+#[derive(Clone, Debug, Deserialize)]
+#[serde(try_from = "DiskSetup")]
+pub struct Disks {
+ pub fs_type: FsType,
+ pub disk_selection: DiskSelection,
+ pub filter_match: Option<FilterMatch>,
+ pub fs_options: FsOptions,
+}
+
+impl TryFrom<DiskSetup> for Disks {
+ type Error = &'static str;
+
+ fn try_from(source: DiskSetup) -> Result<Self, Self::Error> {
+ if source.disk_list.is_empty() && source.filter.is_none() {
+ return Err("Need either 'disk_list' or 'filter' set");
+ }
+ if !source.disk_list.is_empty() && source.filter.is_some() {
+ return Err("Cannot use both, 'disk_list' and 'filter'");
+ }
+
+ let disk_selection = if !source.disk_list.is_empty() {
+ DiskSelection::Selection(source.disk_list.clone())
+ } else {
+ DiskSelection::Filter(source.filter.clone().unwrap())
+ };
+
+ let lvm_checks = |source: &DiskSetup| -> Result<(), Self::Error> {
+ if source.zfs.is_some() || source.btrfs.is_some() {
+ return Err("make sure only 'lvm' options are set");
+ }
+ if source.disk_list.len() > 1 {
+ return Err("make sure to define only one disk for ext4 and xfs");
+ }
+ Ok(())
+ };
+ // TODO: improve checks for foreign FS options. E.g. less verbose and handling new FS types
+ // automatically
+ let (fs, fs_options) = match source.filesystem {
+ Filesystem::Xfs => {
+ lvm_checks(&source)?;
+ (
+ FsType::Xfs,
+ FsOptions::LVM(source.lvm.unwrap_or(LvmOptions::default())),
+ )
+ }
+ Filesystem::Ext4 => {
+ lvm_checks(&source)?;
+ (
+ FsType::Ext4,
+ FsOptions::LVM(source.lvm.unwrap_or(LvmOptions::default())),
+ )
+ }
+ Filesystem::Zfs => {
+ if source.lvm.is_some() || source.btrfs.is_some() {
+ return Err("make sure only 'zfs' options are set");
+ }
+ match source.zfs {
+ None | Some(ZfsOptions { raid: None, .. }) => {
+ return Err("ZFS raid level 'zfs.raid' must be set")
+ }
+ Some(opts) => (FsType::Zfs(opts.raid.unwrap()), FsOptions::ZFS(opts)),
+ }
+ }
+ Filesystem::Btrfs => {
+ if source.zfs.is_some() || source.lvm.is_some() {
+ return Err("make sure only 'btrfs' options are set");
+ }
+ match source.btrfs {
+ None | Some(BtrfsOptions { raid: None, .. }) => {
+ return Err("BTRFS raid level 'btrfs.raid' must be set")
+ }
+ Some(opts) => (FsType::Btrfs(opts.raid.unwrap()), FsOptions::BTRFS(opts)),
+ }
+ }
+ };
+
+ let res = Disks {
+ fs_type: fs,
+ disk_selection,
+ filter_match: source.filter_match,
+ fs_options,
+ };
+ Ok(res)
+ }
+}
+
+#[derive(Clone, Debug)]
+pub enum FsOptions {
+ LVM(LvmOptions),
+ ZFS(ZfsOptions),
+ BTRFS(BtrfsOptions),
+}
+
+#[derive(Clone, Debug)]
+pub enum DiskSelection {
+ Selection(Vec<String>),
+ Filter(BTreeMap<String, String>),
+}
+#[derive(Clone, Deserialize, Debug, PartialEq)]
+#[serde(rename_all = "lowercase")]
+pub enum FilterMatch {
+ Any,
+ All,
+}
+
+#[derive(Clone, Deserialize, Serialize, Debug, PartialEq)]
+#[serde(rename_all = "lowercase")]
+pub enum Filesystem {
+ Ext4,
+ Xfs,
+ Zfs,
+ Btrfs,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Debug)]
+pub struct ZfsOptions {
+ pub raid: Option<ZfsRaidLevel>,
+ pub ashift: Option<usize>,
+ pub arc_max: Option<usize>,
+ pub checksum: Option<ZfsChecksumOption>,
+ pub compress: Option<ZfsCompressOption>,
+ pub copies: Option<usize>,
+ pub hdsize: Option<f64>,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Serialize, Debug)]
+pub struct LvmOptions {
+ pub hdsize: Option<f64>,
+ pub swapsize: Option<f64>,
+ pub maxroot: Option<f64>,
+ pub maxvz: Option<f64>,
+ pub minfree: Option<f64>,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Debug)]
+pub struct BtrfsOptions {
+ pub hdsize: Option<f64>,
+ pub raid: Option<BtrfsRaidLevel>,
+}
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index e69de29..7813b98 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -0,0 +1 @@
+pub mod answer;
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 10/36] auto-installer: add struct to hold udev info
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (8 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 09/36] auto-installer: add answer file definition Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 11/36] auto-installer: add utils Aaron Lauterer
` (25 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/udevinfo.rs | 9 +++++++++
2 files changed, 10 insertions(+)
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 7813b98..8cda416 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1 +1,2 @@
pub mod answer;
+pub mod udevinfo;
diff --git a/proxmox-auto-installer/src/udevinfo.rs b/proxmox-auto-installer/src/udevinfo.rs
new file mode 100644
index 0000000..a6b61b5
--- /dev/null
+++ b/proxmox-auto-installer/src/udevinfo.rs
@@ -0,0 +1,9 @@
+use serde::Deserialize;
+use std::collections::BTreeMap;
+
+#[derive(Clone, Deserialize, Debug)]
+pub struct UdevInfo {
+ // use BTreeMap to have keys sorted
+ pub disks: BTreeMap<String, BTreeMap<String, String>>,
+ pub nics: BTreeMap<String, BTreeMap<String, String>>,
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 11/36] auto-installer: add utils
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (9 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 10/36] auto-installer: add struct to hold udev info Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 12/36] auto-installer: add simple logging Aaron Lauterer
` (24 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
contains several utility structs and functions.
For example: a simple pattern matcher that matches wildcards at the
beginning or end of the filter.
It currently uses a dedicated function (parse_answer) to generate the
InstallConfig struct instead of a From implementation. This is because
for now the source data is spread over several other structs in
comparison to one in the TUI installer.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/utils.rs | 424 ++++++++++++++++++++++++++++
2 files changed, 425 insertions(+)
create mode 100644 proxmox-auto-installer/src/utils.rs
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 8cda416..72884c1 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,2 +1,3 @@
pub mod answer;
pub mod udevinfo;
+pub mod utils;
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
new file mode 100644
index 0000000..ae28b1e
--- /dev/null
+++ b/proxmox-auto-installer/src/utils.rs
@@ -0,0 +1,424 @@
+use anyhow::{bail, Result};
+use log::info;
+use std::{
+ collections::BTreeMap,
+ process::{Command, Stdio},
+};
+
+use crate::{
+ answer::{self, Answer},
+ udevinfo::UdevInfo,
+};
+use proxmox_installer_common::{
+ options::{FsType, NetworkOptions, ZfsChecksumOption, ZfsCompressOption},
+ setup::{InstallConfig, InstallZfsOption, LocaleInfo, RuntimeInfo, SetupInfo},
+};
+use serde::Deserialize;
+
+/// Supports the globbing character '*' at the beginning, end or both of the pattern.
+/// Globbing within the pattern is not supported
+fn find_with_glob(pattern: &str, value: &str) -> bool {
+ let globbing_symbol = '*';
+ let mut start_glob = false;
+ let mut end_glob = false;
+ let mut pattern = pattern;
+
+ if pattern.starts_with(globbing_symbol) {
+ start_glob = true;
+ pattern = &pattern[1..];
+ }
+
+ if pattern.ends_with(globbing_symbol) {
+ end_glob = true;
+ pattern = &pattern[..pattern.len() - 1]
+ }
+
+ match (start_glob, end_glob) {
+ (true, true) => value.contains(pattern),
+ (true, false) => value.ends_with(pattern),
+ (false, true) => value.starts_with(pattern),
+ _ => value == pattern,
+ }
+}
+
+pub fn get_network_settings(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ setup_info: &SetupInfo,
+) -> Result<NetworkOptions> {
+ let mut network_options = NetworkOptions::defaults_from(setup_info, &runtime_info.network);
+
+ info!("Setting network configuration");
+
+ // Always use the FQDN from the answer file
+ network_options.fqdn = answer.global.fqdn.clone();
+
+ if let answer::NetworkSettings::Manual(settings) = &answer.network.network_settings {
+ network_options.address = settings.cidr.clone();
+ network_options.dns_server = settings.dns;
+ network_options.gateway = settings.gateway;
+ network_options.ifname = get_single_udev_index(settings.filter.clone(), &udev_info.nics)?;
+ }
+ info!("Network interface used is '{}'", &network_options.ifname);
+ Ok(network_options)
+}
+
+fn get_single_udev_index(
+ filter: BTreeMap<String, String>,
+ udev_list: &BTreeMap<String, BTreeMap<String, String>>,
+) -> Result<String> {
+ if filter.is_empty() {
+ bail!("no filter defined");
+ }
+ let mut dev_index: Option<String> = None;
+ 'outer: for (dev, dev_values) in udev_list {
+ for (filter_key, filter_value) in &filter {
+ for (udev_key, udev_value) in dev_values {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ dev_index = Some(dev.clone());
+ break 'outer; // take first match
+ }
+ }
+ }
+ }
+ if dev_index.is_none() {
+ bail!("filter did not match any device");
+ }
+
+ Ok(dev_index.unwrap())
+}
+
+fn get_matched_udev_indexes(
+ filter: BTreeMap<String, String>,
+ udev_list: &BTreeMap<String, BTreeMap<String, String>>,
+ match_all: bool,
+) -> Result<Vec<String>> {
+ let mut matches = vec![];
+ for (dev, dev_values) in udev_list {
+ let mut did_match_once = false;
+ let mut did_match_all = true;
+ for (filter_key, filter_value) in &filter {
+ for (udev_key, udev_value) in dev_values {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ did_match_once = true;
+ } else if udev_key == filter_key {
+ did_match_all = false;
+ }
+ }
+ }
+ if (match_all && did_match_all) || (!match_all && did_match_once) {
+ matches.push(dev.clone());
+ }
+ }
+ if matches.is_empty() {
+ bail!("filter did not match any devices");
+ }
+ matches.sort();
+ Ok(matches)
+}
+
+pub fn set_disks(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match config.filesys {
+ FsType::Ext4 | FsType::Xfs => set_single_disk(answer, udev_info, runtime_info, config),
+ FsType::Zfs(_) | FsType::Btrfs(_) => {
+ set_selected_disks(answer, udev_info, runtime_info, config)
+ }
+ }
+}
+
+fn set_single_disk(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match &answer.disks.disk_selection {
+ answer::DiskSelection::Selection(disk_list) => {
+ let disk_name = disk_list[0].clone();
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.path.ends_with(disk_name.as_str()));
+ match disk {
+ Some(disk) => config.target_hd = Some(disk.clone()),
+ None => bail!("disk in 'disk_selection' not found"),
+ }
+ }
+ answer::DiskSelection::Filter(filter) => {
+ let disk_index = get_single_udev_index(filter.clone(), &udev_info.disks)?;
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index == disk_index);
+ config.target_hd = disk.cloned();
+ }
+ }
+ info!("Selected disk: {}", config.target_hd.clone().unwrap().path);
+ Ok(())
+}
+
+fn set_selected_disks(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match &answer.disks.disk_selection {
+ answer::DiskSelection::Selection(disk_list) => {
+ info!("Disk selection found");
+ for disk_name in disk_list.clone() {
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.path.ends_with(disk_name.as_str()));
+ if let Some(disk) = disk {
+ config
+ .disk_selection
+ .insert(disk.index.clone(), disk.index.clone());
+ }
+ }
+ }
+ answer::DiskSelection::Filter(filter) => {
+ info!("No disk list found, looking for disk filters");
+ let filter_match = answer
+ .disks
+ .filter_match
+ .clone()
+ .unwrap_or(answer::FilterMatch::Any);
+ let disk_filters = filter.clone();
+ let selected_disk_indexes = get_matched_udev_indexes(
+ disk_filters,
+ &udev_info.disks,
+ filter_match == answer::FilterMatch::All,
+ )?;
+
+ for i in selected_disk_indexes.into_iter() {
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index == i)
+ .unwrap();
+ config
+ .disk_selection
+ .insert(disk.index.clone(), disk.index.clone());
+ }
+ }
+ }
+ if config.disk_selection.is_empty() {
+ bail!("No disks found matching selection.");
+ }
+
+ let mut selected_disks: Vec<String> = Vec::new();
+ for i in config.disk_selection.keys() {
+ selected_disks.push(
+ runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index.as_str() == i)
+ .unwrap()
+ .clone()
+ .path,
+ );
+ }
+ info!(
+ "Selected disks: {}",
+ selected_disks
+ .iter()
+ .map(|x| x.to_string() + " ")
+ .collect::<String>()
+ );
+
+ Ok(())
+}
+
+pub fn get_first_selected_disk(config: &InstallConfig) -> usize {
+ config
+ .disk_selection
+ .iter()
+ .next()
+ .expect("no disks found")
+ .0
+ .parse::<usize>()
+ .expect("could not parse key to usize")
+}
+
+pub fn verify_locale_settings(answer: &Answer, locales: &LocaleInfo) -> Result<()> {
+ info!("Verifying locale settings");
+ if !locales
+ .countries
+ .keys()
+ .any(|i| i == &answer.global.country)
+ {
+ bail!("country code '{}' is not valid", &answer.global.country);
+ }
+ if !locales.kmap.keys().any(|i| i == &answer.global.keyboard) {
+ bail!("keyboard layout '{}' is not valid", &answer.global.keyboard);
+ }
+ if !locales
+ .cczones
+ .iter()
+ .any(|(_, zones)| zones.contains(&answer.global.timezone))
+ {
+ bail!("timezone '{}' is not valid", &answer.global.timezone);
+ }
+ Ok(())
+}
+
+pub fn run_cmds(step: &str, cmd_vec: &Option<Vec<String>>) -> Result<()> {
+ if let Some(cmds) = cmd_vec {
+ if !cmds.is_empty() {
+ info!("Running {step}-Commands:");
+ run_cmd(cmds)?;
+ info!("{step}-Commands finished");
+ }
+ }
+ Ok(())
+}
+
+fn run_cmd(cmds: &Vec<String>) -> Result<()> {
+ for cmd in cmds {
+ info!("Command '{cmd}':");
+ let child = match Command::new("/bin/bash")
+ .arg("-c")
+ .arg(cmd.clone())
+ .stdout(Stdio::piped())
+ .stderr(Stdio::piped())
+ .spawn()
+ {
+ Ok(child) => child,
+ Err(err) => bail!("error running command {cmd}: {err}"),
+ };
+ match child.wait_with_output() {
+ Ok(output) => {
+ if output.status.success() {
+ info!("{}", String::from_utf8(output.stdout).unwrap());
+ } else {
+ bail!("{}", String::from_utf8(output.stderr).unwrap());
+ }
+ },
+ Err(err) => bail!("{err}"),
+ }
+ }
+
+ Ok(())
+}
+
+pub fn parse_answer(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ locales: &LocaleInfo,
+ setup_info: &SetupInfo,
+) -> Result<InstallConfig> {
+ info!("Parsing answer file");
+ info!("Setting File system");
+ let filesystem = answer.disks.fs_type;
+ info!("File system selected: {}", filesystem);
+
+ let network_settings = get_network_settings(answer, udev_info, runtime_info, setup_info)?;
+
+ verify_locale_settings(answer, locales)?;
+
+ let mut config = InstallConfig {
+ autoreboot: 1_usize,
+ filesys: filesystem,
+ hdsize: 0.,
+ swapsize: None,
+ maxroot: None,
+ minfree: None,
+ maxvz: None,
+ zfs_opts: None,
+ target_hd: None,
+ disk_selection: BTreeMap::new(),
+
+ country: answer.global.country.clone(),
+ timezone: answer.global.timezone.clone(),
+ keymap: answer.global.keyboard.clone(),
+
+ password: answer.global.password.clone(),
+ mailto: answer.global.mailto.clone(),
+
+ mngmt_nic: network_settings.ifname,
+
+ hostname: network_settings.fqdn.host().unwrap().to_string(),
+ domain: network_settings.fqdn.domain(),
+ cidr: network_settings.address,
+ gateway: network_settings.gateway,
+ dns: network_settings.dns_server,
+ };
+
+ set_disks(answer, udev_info, runtime_info, &mut config)?;
+ match &answer.disks.fs_options {
+ answer::FsOptions::LVM(lvm) => {
+ config.hdsize = lvm.hdsize.unwrap_or(config.target_hd.clone().unwrap().size);
+ config.swapsize = lvm.swapsize;
+ config.maxroot = lvm.maxroot;
+ config.maxvz = lvm.maxvz;
+ config.minfree = lvm.minfree;
+ }
+ answer::FsOptions::ZFS(zfs) => {
+ let first_selected_disk = get_first_selected_disk(&config);
+
+ config.hdsize = zfs
+ .hdsize
+ .unwrap_or(runtime_info.disks[first_selected_disk].size);
+ config.zfs_opts = Some(InstallZfsOption {
+ ashift: zfs.ashift.unwrap_or(12),
+ arc_max: zfs.arc_max.unwrap_or(2048),
+ compress: zfs.compress.unwrap_or(ZfsCompressOption::On),
+ checksum: zfs.checksum.unwrap_or(ZfsChecksumOption::On),
+ copies: zfs.copies.unwrap_or(1),
+ });
+ }
+ answer::FsOptions::BTRFS(btrfs) => {
+ let first_selected_disk = get_first_selected_disk(&config);
+
+ config.hdsize = btrfs
+ .hdsize
+ .unwrap_or(runtime_info.disks[first_selected_disk].size);
+ }
+ }
+ Ok(config)
+}
+
+#[derive(Clone, Debug, Deserialize, PartialEq)]
+#[serde(tag = "type", rename_all = "lowercase")]
+pub enum LowLevelMessage {
+ #[serde(rename = "message")]
+ Info {
+ message: String,
+ },
+ Error {
+ message: String,
+ },
+ Prompt {
+ query: String,
+ },
+ Finished {
+ state: String,
+ message: String,
+ },
+ Progress {
+ ratio: f32,
+ text: String,
+ },
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ #[test]
+ fn test_glob_patterns() {
+ let test_value = "foobar";
+ assert_eq!(find_with_glob("*bar", test_value), true);
+ assert_eq!(find_with_glob("foo*", test_value), true);
+ assert_eq!(find_with_glob("foobar", test_value), true);
+ assert_eq!(find_with_glob("oobar", test_value), false);
+ }
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 12/36] auto-installer: add simple logging
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (10 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 11/36] auto-installer: add utils Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 13/36] auto-installer: add tests for answer file parsing Aaron Lauterer
` (23 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Log to stdout and the file the binary needs to set up.
This is a first variant. By using the log crate macros we can change
that in the future without too much effort.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 2 ++
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/log.rs | 38 +++++++++++++++++++++++++++++++
3 files changed, 41 insertions(+)
create mode 100644 proxmox-auto-installer/src/log.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 80de4fa..b1d3e7a 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -8,8 +8,10 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+anyhow = "1.0"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.7"
enum-iterator = "0.6.0"
+log = "0.4.20"
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 72884c1..6636cc7 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,3 +1,4 @@
pub mod answer;
+pub mod log;
pub mod udevinfo;
pub mod utils;
diff --git a/proxmox-auto-installer/src/log.rs b/proxmox-auto-installer/src/log.rs
new file mode 100644
index 0000000..d52912a
--- /dev/null
+++ b/proxmox-auto-installer/src/log.rs
@@ -0,0 +1,38 @@
+use anyhow::{bail, Result};
+use log::{Level, Metadata, Record};
+use std::{fs::File, io::Write, sync::Mutex, sync::OnceLock};
+
+pub struct AutoInstLogger;
+static LOGFILE: OnceLock<Mutex<File>> = OnceLock::new();
+
+impl AutoInstLogger {
+ pub fn init(path: &str) -> Result<()> {
+ let f = File::create(path)?;
+ if LOGFILE.set(Mutex::new(f)).is_err() {
+ bail!("Cannot set LOGFILE")
+ }
+ Ok(())
+ }
+}
+
+impl log::Log for AutoInstLogger {
+ fn enabled(&self, metadata: &Metadata) -> bool {
+ metadata.level() <= Level::Info
+ }
+
+ /// Logs to stdout without log level and into log file including log level
+ fn log(&self, record: &Record) {
+ if self.enabled(record.metadata()) {
+ println!("{}", record.args());
+ let mut file = LOGFILE
+ .get()
+ .expect("could not get LOGFILE")
+ .lock()
+ .expect("could not get mutex for LOGFILE");
+ file.write_all(format!("{} - {}\n", record.level(), record.args()).as_bytes())
+ .expect("could not write to LOGFILE");
+ }
+ }
+
+ fn flush(&self) {}
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 13/36] auto-installer: add tests for answer file parsing
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (11 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 12/36] auto-installer: add simple logging Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 14/36] auto-installer: add auto-installer binary Aaron Lauterer
` (22 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
By matching the resulting json to be passed to the low level installer
against known good ones.
The environment info was gathered from one of our AMD Epyc Rome test
servers to have a realistic starting point.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/tests/parse-answer.rs | 106 ++++++++++++++++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 +++++
.../resources/parse_answer/disk_match.toml | 17 +++
.../parse_answer/disk_match_all.json | 26 +++++
.../parse_answer/disk_match_all.toml | 17 +++
.../parse_answer/disk_match_any.json | 33 ++++++
.../parse_answer/disk_match_any.toml | 17 +++
.../tests/resources/parse_answer/minimal.json | 17 +++
.../tests/resources/parse_answer/minimal.toml | 14 +++
.../resources/parse_answer/nic_matching.json | 17 +++
.../resources/parse_answer/nic_matching.toml | 19 ++++
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +++
.../resources/parse_answer/specific_nic.toml | 19 ++++
.../tests/resources/parse_answer/zfs.json | 27 +++++
.../tests/resources/parse_answer/zfs.toml | 20 ++++
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
20 files changed, 403 insertions(+)
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs
new file mode 100644
index 0000000..c12520f
--- /dev/null
+++ b/proxmox-auto-installer/tests/parse-answer.rs
@@ -0,0 +1,106 @@
+use std::path::PathBuf;
+
+use serde_json::Value;
+use std::fs;
+
+use proxmox_auto_installer::answer;
+use proxmox_auto_installer::answer::Answer;
+use proxmox_auto_installer::udevinfo::UdevInfo;
+use proxmox_auto_installer::utils::parse_answer;
+
+use proxmox_installer_common::setup::{read_json, LocaleInfo, RuntimeInfo, SetupInfo};
+
+fn get_test_resource_path() -> Result<PathBuf, String> {
+ Ok(std::env::current_dir()
+ .expect("current dir failed")
+ .join("tests/resources"))
+}
+fn get_answer(path: PathBuf) -> Result<Answer, String> {
+ let answer_raw = std::fs::read_to_string(&path).unwrap();
+ let answer: answer::Answer = toml::from_str(&answer_raw)
+ .map_err(|err| format!("error parsing answer.toml: {err}"))
+ .unwrap();
+
+ Ok(answer)
+}
+
+fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
+ let installer_info: SetupInfo = {
+ let mut path = path.clone();
+ path.push("iso-info.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve setup info: {err}"))
+ .unwrap()
+ };
+
+ let locale_info = {
+ let mut path = path.clone();
+ path.push("locales.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve locale info: {err}"))
+ .unwrap()
+ };
+
+ let mut runtime_info: RuntimeInfo = {
+ let mut path = path.clone();
+ path.push("run-env-info.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve runtime environment info: {err}"))
+ .unwrap()
+ };
+
+ let udev_info: UdevInfo = {
+ let mut path = path.clone();
+ path.push("run-env-udev.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve udev info details: {err}"))
+ .unwrap()
+ };
+ runtime_info.disks.sort();
+ if runtime_info.disks.is_empty() {
+ panic!("disk list is empty!");
+ }
+ (installer_info, locale_info, runtime_info, udev_info)
+}
+
+#[test]
+fn test_parse_answers() {
+ let path = get_test_resource_path().unwrap();
+ let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path);
+ let mut tests_path = path.clone();
+ tests_path.push("parse_answer");
+ let test_dir = fs::read_dir(tests_path.clone()).unwrap();
+
+ for file_entry in test_dir {
+ let file = file_entry.unwrap();
+ if !file.file_type().unwrap().is_file() || file.file_name() == "readme" {
+ continue;
+ }
+ let p = file.path();
+ let name = p.file_stem().unwrap().to_str().unwrap();
+ let extension = p.extension().unwrap().to_str().unwrap();
+ if extension == "toml" {
+ println!("Test: {name}");
+ let answer = get_answer(p.clone()).unwrap();
+ let config =
+ &parse_answer(&answer, &udev_info, &runtime_info, &locales, &setup_info).unwrap();
+ println!("Selected disks: {:#?}", &config.disk_selection);
+ let config_json = serde_json::to_string(config);
+ let config: Value = serde_json::from_str(config_json.unwrap().as_str()).unwrap();
+ let mut path = tests_path.clone();
+ path.push(format!("{name}.json"));
+ let compare_raw = std::fs::read_to_string(&path).unwrap();
+ let compare: Value = serde_json::from_str(&compare_raw).unwrap();
+ if config != compare {
+ panic!(
+ "Test {} failed:\nleft: {:#?}\nright: {:#?}\n",
+ name, config, compare
+ );
+ }
+ }
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/iso-info.json b/proxmox-auto-installer/tests/resources/iso-info.json
new file mode 100644
index 0000000..33cb79b
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/iso-info.json
@@ -0,0 +1 @@
+{"iso-info":{"isoname":"proxmox-ve","isorelease":"2","product":"pve","productlong":"Proxmox VE","release":"8.0"},"locations":{"iso":"/cdrom","lib":"/var/lib/proxmox-installer","pkg":"/cdrom/proxmox/packages/","run":"/run/proxmox-installer"},"product":"pve","product-cfg":{"bridged_network":1,"enable_btrfs":1,"fullname":"Proxmox VE","port":"8006","product":"pve"},"run-env-cache-file":"/run/proxmox-installer/run-env-info.json"}
diff --git a/proxmox-auto-installer/tests/resources/locales.json b/proxmox-auto-installer/tests/resources/locales.json
new file mode 100644
index 0000000..220a18c
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/locales.json
@@ -0,0 +1 @@
+{"cczones":{"ad":{"Europe/Andorra":1},"ae":{"Asia/Dubai":1},"af":{"Asia/Kabul":1},"ag":{"America/Antigua":1},"ai":{"America/Anguilla":1},"al":{"Europe/Tirane":1},"am":{"Asia/Yerevan":1},"ao":{"Africa/Luanda":1},"aq":{"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1},"ar":{"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1},"as":{"Pacific/Pago_Pago":1},"at":{"Europe/Vienna":1},"au":{"Antarctica/Macquarie":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1},"aw":{"America/Aruba":1},"ax":{"Europe/Mariehamn":1},"az":{"Asia/Baku":1},"ba":{"Europe/Sarajevo":1},"bb":{"America/Barbados":1},"bd":{"Asia/Dhaka":1},"be":{"Europe/Brussels":1},"bf":{"Africa/Ouagadougou":1},"bg":{"Europe/Sofia":1},"bh":{"Asia/Bahrain":1},"bi":{"Africa/Bujumbura":1},"bj":{"Africa/Porto-Novo":1},"bl":{"America/St_Barthelemy":1},"bm":{"Atlantic/Bermuda":1},"bn":{"Asia/Brunei":1},"bo":{"America/La_Paz":1},"bq":{"America/Kralendijk":1},"br":{"America/Araguaina":1,"America/Bahia":1,"America/Belem":1,"America/Boa_Vista":1,"America/Campo_Grande":1,"America/Cuiaba":1,"America/Eirunepe":1,"America/Fortaleza":1,"America/Maceio":1,"America/Manaus":1,"America/Noronha":1,"America/Porto_Velho":1,"America/Recife":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Sao_Paulo":1},"bs":{"America/Nassau":1},"bt":{"Asia/Thimphu":1},"bw":{"Africa/Gaborone":1},"by":{"Europe/Minsk":1},"bz":{"America/Belize":1},"ca":{"America/Atikokan":1,"America/Blanc-Sablon":1,"America/Cambridge_Bay":1,"America/Creston":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Edmonton":1,"America/Fort_Nelson":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Halifax":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Moncton":1,"America/Rankin_Inlet":1,"America/Regina":1,"America/Resolute":1,"America/St_Johns":1,"America/Swift_Current":1,"America/Toronto":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1},"cc":{"Indian/Cocos":1},"cd":{"Africa/Kinshasa":1,"Africa/Lubumbashi":1},"cf":{"Africa/Bangui":1},"cg":{"Africa/Brazzaville":1},"ch":{"Europe/Zurich":1},"ci":{"Africa/Abidjan":1},"ck":{"Pacific/Rarotonga":1},"cl":{"America/Punta_Arenas":1,"America/Santiago":1,"Pacific/Easter":1},"cm":{"Africa/Douala":1},"cn":{"Asia/Shanghai":1,"Asia/Urumqi":1},"co":{"America/Bogota":1},"cr":{"America/Costa_Rica":1},"cu":{"America/Havana":1},"cv":{"Atlantic/Cape_Verde":1},"cw":{"America/Curacao":1},"cx":{"Indian/Christmas":1},"cy":{"Asia/Famagusta":1,"Asia/Nicosia":1},"cz":{"Europe/Prague":1},"de":{"Europe/Berlin":1,"Europe/Busingen":1},"dj":{"Africa/Djibouti":1},"dk":{"Europe/Copenhagen":1},"dm":{"America/Dominica":1},"do":{"America/Santo_Domingo":1},"dz":{"Africa/Algiers":1},"ec":{"America/Guayaquil":1,"Pacific/Galapagos":1},"ee":{"Europe/Tallinn":1},"eg":{"Africa/Cairo":1},"eh":{"Africa/El_Aaiun":1},"er":{"Africa/Asmara":1},"es":{"Africa/Ceuta":1,"Atlantic/Canary":1,"Europe/Madrid":1},"et":{"Africa/Addis_Ababa":1},"fi":{"Europe/Helsinki":1},"fj":{"Pacific/Fiji":1},"fk":{"Atlantic/Stanley":1},"fm":{"Pacific/Chuuk":1,"Pacific/Kosrae":1,"Pacific/Pohnpei":1},"fo":{"Atlantic/Faroe":1},"fr":{"Europe/Paris":1},"ga":{"Africa/Libreville":1},"gb":{"Europe/London":1},"gd":{"America/Grenada":1},"ge":{"Asia/Tbilisi":1},"gf":{"America/Cayenne":1},"gg":{"Europe/Guernsey":1},"gh":{"Africa/Accra":1},"gi":{"Europe/Gibraltar":1},"gl":{"America/Danmarkshavn":1,"America/Nuuk":1,"America/Scoresbysund":1,"America/Thule":1},"gm":{"Africa/Banjul":1},"gn":{"Africa/Conakry":1},"gp":{"America/Guadeloupe":1},"gq":{"Africa/Malabo":1},"gr":{"Europe/Athens":1},"gs":{"Atlantic/South_Georgia":1},"gt":{"America/Guatemala":1},"gu":{"Pacific/Guam":1},"gw":{"Africa/Bissau":1},"gy":{"America/Guyana":1},"hk":{"Asia/Hong_Kong":1},"hn":{"America/Tegucigalpa":1},"hr":{"Europe/Zagreb":1},"ht":{"America/Port-au-Prince":1},"hu":{"Europe/Budapest":1},"id":{"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Makassar":1,"Asia/Pontianak":1},"ie":{"Europe/Dublin":1},"il":{"Asia/Jerusalem":1},"im":{"Europe/Isle_of_Man":1},"in":{"Asia/Kolkata":1},"io":{"Indian/Chagos":1},"iq":{"Asia/Baghdad":1},"ir":{"Asia/Tehran":1},"is":{"Atlantic/Reykjavik":1},"it":{"Europe/Rome":1},"je":{"Europe/Jersey":1},"jm":{"America/Jamaica":1},"jo":{"Asia/Amman":1},"jp":{"Asia/Tokyo":1},"ke":{"Africa/Nairobi":1},"kg":{"Asia/Bishkek":1},"kh":{"Asia/Phnom_Penh":1},"ki":{"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Tarawa":1},"km":{"Indian/Comoro":1},"kn":{"America/St_Kitts":1},"kp":{"Asia/Pyongyang":1},"kr":{"Asia/Seoul":1},"kw":{"Asia/Kuwait":1},"ky":{"America/Cayman":1},"kz":{"Asia/Almaty":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Atyrau":1,"Asia/Oral":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1},"la":{"Asia/Vientiane":1},"lb":{"Asia/Beirut":1},"lc":{"America/St_Lucia":1},"li":{"Europe/Vaduz":1},"lk":{"Asia/Colombo":1},"lr":{"Africa/Monrovia":1},"ls":{"Africa/Maseru":1},"lt":{"Europe/Vilnius":1},"lu":{"Europe/Luxembourg":1},"lv":{"Europe/Riga":1},"ly":{"Africa/Tripoli":1},"ma":{"Africa/Casablanca":1},"mc":{"Europe/Monaco":1},"md":{"Europe/Chisinau":1},"me":{"Europe/Podgorica":1},"mf":{"America/Marigot":1},"mg":{"Indian/Antananarivo":1},"mh":{"Pacific/Kwajalein":1,"Pacific/Majuro":1},"mk":{"Europe/Skopje":1},"ml":{"Africa/Bamako":1},"mm":{"Asia/Yangon":1},"mn":{"Asia/Choibalsan":1,"Asia/Hovd":1,"Asia/Ulaanbaatar":1},"mo":{"Asia/Macau":1},"mp":{"Pacific/Saipan":1},"mq":{"America/Martinique":1},"mr":{"Africa/Nouakchott":1},"ms":{"America/Montserrat":1},"mt":{"Europe/Malta":1},"mu":{"Indian/Mauritius":1},"mv":{"Indian/Maldives":1},"mw":{"Africa/Blantyre":1},"mx":{"America/Bahia_Banderas":1,"America/Cancun":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Hermosillo":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Merida":1,"America/Mexico_City":1,"America/Monterrey":1,"America/Ojinaga":1,"America/Tijuana":1},"my":{"Asia/Kuala_Lumpur":1,"Asia/Kuching":1},"mz":{"Africa/Maputo":1},"na":{"Africa/Windhoek":1},"nc":{"Pacific/Noumea":1},"ne":{"Africa/Niamey":1},"nf":{"Pacific/Norfolk":1},"ng":{"Africa/Lagos":1},"ni":{"America/Managua":1},"nl":{"Europe/Amsterdam":1},"no":{"Europe/Oslo":1},"np":{"Asia/Kathmandu":1},"nr":{"Pacific/Nauru":1},"nu":{"Pacific/Niue":1},"nz":{"Pacific/Auckland":1,"Pacific/Chatham":1},"om":{"Asia/Muscat":1},"pa":{"America/Panama":1},"pe":{"America/Lima":1},"pf":{"Pacific/Gambier":1,"Pacific/Marquesas":1,"Pacific/Tahiti":1},"pg":{"Pacific/Bougainville":1,"Pacific/Port_Moresby":1},"ph":{"Asia/Manila":1},"pk":{"Asia/Karachi":1},"pl":{"Europe/Warsaw":1},"pm":{"America/Miquelon":1},"pn":{"Pacific/Pitcairn":1},"pr":{"America/Puerto_Rico":1},"ps":{"Asia/Gaza":1,"Asia/Hebron":1},"pt":{"Atlantic/Azores":1,"Atlantic/Madeira":1,"Europe/Lisbon":1},"pw":{"Pacific/Palau":1},"py":{"America/Asuncion":1},"qa":{"Asia/Qatar":1},"re":{"Indian/Reunion":1},"ro":{"Europe/Bucharest":1},"rs":{"Europe/Belgrade":1},"ru":{"Asia/Anadyr":1,"Asia/Barnaul":1,"Asia/Chita":1,"Asia/Irkutsk":1,"Asia/Kamchatka":1,"Asia/Khandyga":1,"Asia/Krasnoyarsk":1,"Asia/Magadan":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Sakhalin":1,"Asia/Srednekolymsk":1,"Asia/Tomsk":1,"Asia/Ust-Nera":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yekaterinburg":1,"Europe/Astrakhan":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Moscow":1,"Europe/Samara":1,"Europe/Saratov":1,"Europe/Ulyanovsk":1,"Europe/Volgograd":1},"rw":{"Africa/Kigali":1},"sa":{"Asia/Riyadh":1},"sb":{"Pacific/Guadalcanal":1},"sc":{"Indian/Mahe":1},"sd":{"Africa/Khartoum":1},"se":{"Europe/Stockholm":1},"sg":{"Asia/Singapore":1},"sh":{"Atlantic/St_Helena":1},"si":{"Europe/Ljubljana":1},"sj":{"Arctic/Longyearbyen":1},"sk":{"Europe/Bratislava":1},"sl":{"Africa/Freetown":1},"sm":{"Europe/San_Marino":1},"sn":{"Africa/Dakar":1},"so":{"Africa/Mogadishu":1},"sr":{"America/Paramaribo":1},"ss":{"Africa/Juba":1},"st":{"Africa/Sao_Tome":1},"sv":{"America/El_Salvador":1},"sx":{"America/Lower_Princes":1},"sy":{"Asia/Damascus":1},"sz":{"Africa/Mbabane":1},"tc":{"America/Grand_Turk":1},"td":{"Africa/Ndjamena":1},"tf":{"Indian/Kerguelen":1},"tg":{"Africa/Lome":1},"th":{"Asia/Bangkok":1},"tj":{"Asia/Dushanbe":1},"tk":{"Pacific/Fakaofo":1},"tl":{"Asia/Dili":1},"tm":{"Asia/Ashgabat":1},"tn":{"Africa/Tunis":1},"to":{"Pacific/Tongatapu":1},"tr":{"Europe/Istanbul":1},"tt":{"America/Port_of_Spain":1},"tv":{"Pacific/Funafuti":1},"tw":{"Asia/Taipei":1},"tz":{"Africa/Dar_es_Salaam":1},"ua":{"Europe/Kyiv":1,"Europe/Simferopol":1},"ug":{"Africa/Kampala":1},"um":{"Pacific/Midway":1,"Pacific/Wake":1},"us":{"America/Adak":1,"America/Anchorage":1,"America/Boise":1,"America/Chicago":1,"America/Denver":1,"America/Detroit":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Los_Angeles":1,"America/Menominee":1,"America/Metlakatla":1,"America/New_York":1,"America/Nome":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Phoenix":1,"America/Sitka":1,"America/Yakutat":1,"Pacific/Honolulu":1},"uy":{"America/Montevideo":1},"uz":{"Asia/Samarkand":1,"Asia/Tashkent":1},"va":{"Europe/Vatican":1},"vc":{"America/St_Vincent":1},"ve":{"America/Caracas":1},"vg":{"America/Tortola":1},"vi":{"America/St_Thomas":1},"vn":{"Asia/Ho_Chi_Minh":1},"vu":{"Pacific/Efate":1},"wf":{"Pacific/Wallis":1},"ws":{"Pacific/Apia":1},"ye":{"Asia/Aden":1},"yt":{"Indian/Mayotte":1},"za":{"Africa/Johannesburg":1},"zm":{"Africa/Lusaka":1},"zw":{"Africa/Harare":1}},"country":{"ad":{"kmap":"","mirror":"","name":"Andorra","zone":"Europe/Andorra"},"ae":{"kmap":"","mirror":"","name":"United Arab Emirates","zone":"Asia/Dubai"},"af":{"kmap":"","mirror":"","name":"Afghanistan","zone":"Asia/Kabul"},"ag":{"kmap":"","mirror":"","name":"Antigua and Barbuda","zone":"America/Antigua"},"ai":{"kmap":"","mirror":"","name":"Anguilla","zone":"America/Anguilla"},"al":{"kmap":"","mirror":"","name":"Albania","zone":"Europe/Tirane"},"am":{"kmap":"","mirror":"","name":"Armenia","zone":"Asia/Yerevan"},"ao":{"kmap":"","mirror":"","name":"Angola","zone":"Africa/Luanda"},"aq":{"kmap":"","mirror":"","name":"Antarctica","zone":"Antarctica/McMurdo"},"ar":{"kmap":"","mirror":"","name":"Argentina","zone":"America/Argentina/Buenos_Aires"},"as":{"kmap":"","mirror":"","name":"American Samoa","zone":"Pacific/Pago_Pago"},"at":{"kmap":"de","mirror":"ftp.at.debian.org","name":"Austria","zone":"Europe/Vienna"},"au":{"kmap":"","mirror":"ftp.au.debian.org","name":"Australia","zone":"Australia/Lord_Howe"},"aw":{"kmap":"","mirror":"","name":"Aruba","zone":"America/Aruba"},"ax":{"kmap":"","mirror":"","name":"Åland Islands","zone":"Europe/Mariehamn"},"az":{"kmap":"","mirror":"","name":"Azerbaijan","zone":"Asia/Baku"},"ba":{"kmap":"","mirror":"","name":"Bosnia and Herzegovina","zone":"Europe/Sarajevo"},"bb":{"kmap":"","mirror":"","name":"Barbados","zone":"America/Barbados"},"bd":{"kmap":"","mirror":"","name":"Bangladesh","zone":"Asia/Dhaka"},"be":{"kmap":"fr-be","mirror":"ftp.be.debian.org","name":"Belgium","zone":"Europe/Brussels"},"bf":{"kmap":"","mirror":"","name":"Burkina Faso","zone":"Africa/Ouagadougou"},"bg":{"kmap":"","mirror":"ftp.bg.debian.org","name":"Bulgaria","zone":"Europe/Sofia"},"bh":{"kmap":"","mirror":"","name":"Bahrain","zone":"Asia/Bahrain"},"bi":{"kmap":"","mirror":"","name":"Burundi","zone":"Africa/Bujumbura"},"bj":{"kmap":"","mirror":"","name":"Benin","zone":"Africa/Porto-Novo"},"bl":{"kmap":"","mirror":"","name":"Saint Barthélemy","zone":"America/St_Barthelemy"},"bm":{"kmap":"","mirror":"","name":"Bermuda","zone":"Atlantic/Bermuda"},"bn":{"kmap":"","mirror":"","name":"Brunei Darussalam","zone":"Asia/Brunei"},"bo":{"kmap":"","mirror":"","name":"Bolivia","zone":"America/La_Paz"},"bq":{"kmap":"","mirror":"","name":"Bonaire, Sint Eustatius and Saba","zone":"America/Kralendijk"},"br":{"kmap":"pt-br","mirror":"ftp.br.debian.org","name":"Brazil","zone":"America/Noronha"},"bs":{"kmap":"","mirror":"","name":"Bahamas","zone":"America/Nassau"},"bt":{"kmap":"","mirror":"","name":"Bhutan","zone":"Asia/Thimphu"},"bv":{"kmap":"","mirror":"","name":"Bouvet Island"},"bw":{"kmap":"","mirror":"","name":"Botswana","zone":"Africa/Gaborone"},"by":{"kmap":"","mirror":"","name":"Belarus","zone":"Europe/Minsk"},"bz":{"kmap":"","mirror":"","name":"Belize","zone":"America/Belize"},"ca":{"kmap":"en-us","mirror":"ftp.ca.debian.org","name":"Canada","zone":"America/St_Johns"},"cc":{"kmap":"","mirror":"","name":"Cocos (Keeling) Islands","zone":"Indian/Cocos"},"cd":{"kmap":"","mirror":"","name":"Congo, The Democratic Republic of the","zone":"Africa/Kinshasa"},"cf":{"kmap":"","mirror":"","name":"Central African Republic","zone":"Africa/Bangui"},"cg":{"kmap":"","mirror":"","name":"Congo","zone":"Africa/Brazzaville"},"ch":{"kmap":"de-ch","mirror":"ftp.ch.debian.org","name":"Switzerland","zone":"Europe/Zurich"},"ci":{"kmap":"","mirror":"","name":"Côte d'Ivoire","zone":"Africa/Abidjan"},"ck":{"kmap":"","mirror":"","name":"Cook Islands","zone":"Pacific/Rarotonga"},"cl":{"kmap":"","mirror":"ftp.cl.debian.org","name":"Chile","zone":"America/Santiago"},"cm":{"kmap":"","mirror":"","name":"Cameroon","zone":"Africa/Douala"},"cn":{"kmap":"","mirror":"","name":"China","zone":"Asia/Shanghai"},"co":{"kmap":"","mirror":"","name":"Colombia","zone":"America/Bogota"},"cr":{"kmap":"","mirror":"","name":"Costa Rica","zone":"America/Costa_Rica"},"cu":{"kmap":"","mirror":"","name":"Cuba","zone":"America/Havana"},"cv":{"kmap":"","mirror":"","name":"Cabo Verde","zone":"Atlantic/Cape_Verde"},"cw":{"kmap":"","mirror":"","name":"Curaçao","zone":"America/Curacao"},"cx":{"kmap":"","mirror":"","name":"Christmas Island","zone":"Indian/Christmas"},"cy":{"kmap":"","mirror":"","name":"Cyprus","zone":"Asia/Nicosia"},"cz":{"kmap":"","mirror":"ftp.cz.debian.org","name":"Czechia","zone":"Europe/Prague"},"de":{"kmap":"de","mirror":"ftp.de.debian.org","name":"Germany","zone":"Europe/Berlin"},"dj":{"kmap":"","mirror":"","name":"Djibouti","zone":"Africa/Djibouti"},"dk":{"kmap":"dk","mirror":"ftp.dk.debian.org","name":"Denmark","zone":"Europe/Copenhagen"},"dm":{"kmap":"","mirror":"","name":"Dominica","zone":"America/Dominica"},"do":{"kmap":"","mirror":"","name":"Dominican Republic","zone":"America/Santo_Domingo"},"dz":{"kmap":"","mirror":"","name":"Algeria","zone":"Africa/Algiers"},"ec":{"kmap":"","mirror":"","name":"Ecuador","zone":"America/Guayaquil"},"ee":{"kmap":"","mirror":"ftp.ee.debian.org","name":"Estonia","zone":"Europe/Tallinn"},"eg":{"kmap":"","mirror":"","name":"Egypt","zone":"Africa/Cairo"},"eh":{"kmap":"","mirror":"","name":"Western Sahara","zone":"Africa/El_Aaiun"},"er":{"kmap":"","mirror":"","name":"Eritrea","zone":"Africa/Asmara"},"es":{"kmap":"es","mirror":"ftp.es.debian.org","name":"Spain","zone":"Europe/Madrid"},"et":{"kmap":"","mirror":"","name":"Ethiopia","zone":"Africa/Addis_Ababa"},"fi":{"kmap":"fi","mirror":"ftp.fi.debian.org","name":"Finland","zone":"Europe/Helsinki"},"fj":{"kmap":"","mirror":"","name":"Fiji","zone":"Pacific/Fiji"},"fk":{"kmap":"","mirror":"","name":"Falkland Islands (Malvinas)","zone":"Atlantic/Stanley"},"fm":{"kmap":"","mirror":"","name":"Micronesia, Federated States of","zone":"Pacific/Chuuk"},"fo":{"kmap":"","mirror":"","name":"Faroe Islands","zone":"Atlantic/Faroe"},"fr":{"kmap":"fr","mirror":"ftp.fr.debian.org","name":"France","zone":"Europe/Paris"},"ga":{"kmap":"","mirror":"","name":"Gabon","zone":"Africa/Libreville"},"gb":{"kmap":"en-gb","mirror":"ftp.uk.debian.org","name":"United Kingdom","zone":"Europe/London"},"gd":{"kmap":"","mirror":"","name":"Grenada","zone":"America/Grenada"},"ge":{"kmap":"","mirror":"","name":"Georgia","zone":"Asia/Tbilisi"},"gf":{"kmap":"","mirror":"","name":"French Guiana","zone":"America/Cayenne"},"gg":{"kmap":"","mirror":"","name":"Guernsey","zone":"Europe/Guernsey"},"gh":{"kmap":"","mirror":"","name":"Ghana","zone":"Africa/Accra"},"gi":{"kmap":"es","mirror":"","name":"Gibraltar","zone":"Europe/Gibraltar"},"gl":{"kmap":"","mirror":"","name":"Greenland","zone":"America/Nuuk"},"gm":{"kmap":"","mirror":"","name":"Gambia","zone":"Africa/Banjul"},"gn":{"kmap":"","mirror":"","name":"Guinea","zone":"Africa/Conakry"},"gp":{"kmap":"","mirror":"","name":"Guadeloupe","zone":"America/Guadeloupe"},"gq":{"kmap":"","mirror":"","name":"Equatorial Guinea","zone":"Africa/Malabo"},"gr":{"kmap":"","mirror":"ftp.gr.debian.org","name":"Greece","zone":"Europe/Athens"},"gs":{"kmap":"","mirror":"","name":"South Georgia and the South Sandwich Islands","zone":"Atlantic/South_Georgia"},"gt":{"kmap":"","mirror":"","name":"Guatemala","zone":"America/Guatemala"},"gu":{"kmap":"","mirror":"","name":"Guam","zone":"Pacific/Guam"},"gw":{"kmap":"","mirror":"","name":"Guinea-Bissau","zone":"Africa/Bissau"},"gy":{"kmap":"","mirror":"","name":"Guyana","zone":"America/Guyana"},"hk":{"kmap":"","mirror":"ftp.hk.debian.org","name":"Hong Kong","zone":"Asia/Hong_Kong"},"hm":{"kmap":"","mirror":"","name":"Heard Island and McDonald Islands"},"hn":{"kmap":"","mirror":"","name":"Honduras","zone":"America/Tegucigalpa"},"hr":{"kmap":"","mirror":"ftp.hr.debian.org","name":"Croatia","zone":"Europe/Zagreb"},"ht":{"kmap":"","mirror":"","name":"Haiti","zone":"America/Port-au-Prince"},"hu":{"kmap":"hu","mirror":"ftp.hu.debian.org","name":"Hungary","zone":"Europe/Budapest"},"id":{"kmap":"","mirror":"","name":"Indonesia","zone":"Asia/Jakarta"},"ie":{"kmap":"","mirror":"ftp.ie.debian.org","name":"Ireland","zone":"Europe/Dublin"},"il":{"kmap":"","mirror":"","name":"Israel","zone":"Asia/Jerusalem"},"im":{"kmap":"","mirror":"","name":"Isle of Man","zone":"Europe/Isle_of_Man"},"in":{"kmap":"","mirror":"","name":"India","zone":"Asia/Kolkata"},"io":{"kmap":"","mirror":"","name":"British Indian Ocean Territory","zone":"Indian/Chagos"},"iq":{"kmap":"","mirror":"","name":"Iraq","zone":"Asia/Baghdad"},"ir":{"kmap":"","mirror":"","name":"Iran","zone":"Asia/Tehran"},"is":{"kmap":"is","mirror":"ftp.is.debian.org","name":"Iceland","zone":"Atlantic/Reykjavik"},"it":{"kmap":"it","mirror":"ftp.it.debian.org","name":"Italy","zone":"Europe/Rome"},"je":{"kmap":"","mirror":"","name":"Jersey","zone":"Europe/Jersey"},"jm":{"kmap":"","mirror":"","name":"Jamaica","zone":"America/Jamaica"},"jo":{"kmap":"","mirror":"","name":"Jordan","zone":"Asia/Amman"},"jp":{"kmap":"jp","mirror":"ftp.jp.debian.org","name":"Japan","zone":"Asia/Tokyo"},"ke":{"kmap":"","mirror":"","name":"Kenya","zone":"Africa/Nairobi"},"kg":{"kmap":"","mirror":"","name":"Kyrgyzstan","zone":"Asia/Bishkek"},"kh":{"kmap":"","mirror":"","name":"Cambodia","zone":"Asia/Phnom_Penh"},"ki":{"kmap":"","mirror":"","name":"Kiribati","zone":"Pacific/Tarawa"},"km":{"kmap":"","mirror":"","name":"Comoros","zone":"Indian/Comoro"},"kn":{"kmap":"","mirror":"","name":"Saint Kitts and Nevis","zone":"America/St_Kitts"},"kp":{"kmap":"","mirror":"","name":"North Korea","zone":"Asia/Pyongyang"},"kr":{"kmap":"","mirror":"ftp.kr.debian.org","name":"South Korea","zone":"Asia/Seoul"},"kw":{"kmap":"","mirror":"","name":"Kuwait","zone":"Asia/Kuwait"},"ky":{"kmap":"","mirror":"","name":"Cayman Islands","zone":"America/Cayman"},"kz":{"kmap":"","mirror":"","name":"Kazakhstan","zone":"Asia/Almaty"},"la":{"kmap":"","mirror":"","name":"Laos","zone":"Asia/Vientiane"},"lb":{"kmap":"","mirror":"","name":"Lebanon","zone":"Asia/Beirut"},"lc":{"kmap":"","mirror":"","name":"Saint Lucia","zone":"America/St_Lucia"},"li":{"kmap":"de-ch","mirror":"","name":"Liechtenstein","zone":"Europe/Vaduz"},"lk":{"kmap":"","mirror":"","name":"Sri Lanka","zone":"Asia/Colombo"},"lr":{"kmap":"","mirror":"","name":"Liberia","zone":"Africa/Monrovia"},"ls":{"kmap":"","mirror":"","name":"Lesotho","zone":"Africa/Maseru"},"lt":{"kmap":"lt","mirror":"","name":"Lithuania","zone":"Europe/Vilnius"},"lu":{"kmap":"fr-ch","mirror":"","name":"Luxembourg","zone":"Europe/Luxembourg"},"lv":{"kmap":"","mirror":"","name":"Latvia","zone":"Europe/Riga"},"ly":{"kmap":"","mirror":"","name":"Libya","zone":"Africa/Tripoli"},"ma":{"kmap":"","mirror":"","name":"Morocco","zone":"Africa/Casablanca"},"mc":{"kmap":"","mirror":"","name":"Monaco","zone":"Europe/Monaco"},"md":{"kmap":"","mirror":"","name":"Moldova","zone":"Europe/Chisinau"},"me":{"kmap":"","mirror":"","name":"Montenegro","zone":"Europe/Podgorica"},"mf":{"kmap":"","mirror":"","name":"Saint Martin (French part)","zone":"America/Marigot"},"mg":{"kmap":"","mirror":"","name":"Madagascar","zone":"Indian/Antananarivo"},"mh":{"kmap":"","mirror":"","name":"Marshall Islands","zone":"Pacific/Majuro"},"mk":{"kmap":"mk","mirror":"","name":"North Macedonia","zone":"Europe/Skopje"},"ml":{"kmap":"","mirror":"","name":"Mali","zone":"Africa/Bamako"},"mm":{"kmap":"","mirror":"","name":"Myanmar","zone":"Asia/Yangon"},"mn":{"kmap":"","mirror":"","name":"Mongolia","zone":"Asia/Ulaanbaatar"},"mo":{"kmap":"","mirror":"","name":"Macao","zone":"Asia/Macau"},"mp":{"kmap":"","mirror":"","name":"Northern Mariana Islands","zone":"Pacific/Saipan"},"mq":{"kmap":"","mirror":"","name":"Martinique","zone":"America/Martinique"},"mr":{"kmap":"","mirror":"","name":"Mauritania","zone":"Africa/Nouakchott"},"ms":{"kmap":"","mirror":"","name":"Montserrat","zone":"America/Montserrat"},"mt":{"kmap":"","mirror":"","name":"Malta","zone":"Europe/Malta"},"mu":{"kmap":"","mirror":"","name":"Mauritius","zone":"Indian/Mauritius"},"mv":{"kmap":"","mirror":"","name":"Maldives","zone":"Indian/Maldives"},"mw":{"kmap":"","mirror":"","name":"Malawi","zone":"Africa/Blantyre"},"mx":{"kmap":"","mirror":"ftp.mx.debian.org","name":"Mexico","zone":"America/Mexico_City"},"my":{"kmap":"","mirror":"","name":"Malaysia","zone":"Asia/Kuala_Lumpur"},"mz":{"kmap":"","mirror":"","name":"Mozambique","zone":"Africa/Maputo"},"na":{"kmap":"","mirror":"","name":"Namibia","zone":"Africa/Windhoek"},"nc":{"kmap":"","mirror":"","name":"New Caledonia","zone":"Pacific/Noumea"},"ne":{"kmap":"","mirror":"","name":"Niger","zone":"Africa/Niamey"},"nf":{"kmap":"","mirror":"","name":"Norfolk Island","zone":"Pacific/Norfolk"},"ng":{"kmap":"","mirror":"","name":"Nigeria","zone":"Africa/Lagos"},"ni":{"kmap":"","mirror":"","name":"Nicaragua","zone":"America/Managua"},"nl":{"kmap":"en-us","mirror":"ftp.nl.debian.org","name":"Netherlands","zone":"Europe/Amsterdam"},"no":{"kmap":"no","mirror":"ftp.no.debian.org","name":"Norway","zone":"Europe/Oslo"},"np":{"kmap":"","mirror":"","name":"Nepal","zone":"Asia/Kathmandu"},"nr":{"kmap":"","mirror":"","name":"Nauru","zone":"Pacific/Nauru"},"nu":{"kmap":"","mirror":"","name":"Niue","zone":"Pacific/Niue"},"nz":{"kmap":"","mirror":"ftp.nz.debian.org","name":"New Zealand","zone":"Pacific/Auckland"},"om":{"kmap":"","mirror":"","name":"Oman","zone":"Asia/Muscat"},"pa":{"kmap":"","mirror":"","name":"Panama","zone":"America/Panama"},"pe":{"kmap":"","mirror":"","name":"Peru","zone":"America/Lima"},"pf":{"kmap":"","mirror":"","name":"French Polynesia","zone":"Pacific/Tahiti"},"pg":{"kmap":"","mirror":"","name":"Papua New Guinea","zone":"Pacific/Port_Moresby"},"ph":{"kmap":"","mirror":"","name":"Philippines","zone":"Asia/Manila"},"pk":{"kmap":"","mirror":"","name":"Pakistan","zone":"Asia/Karachi"},"pl":{"kmap":"pl","mirror":"ftp.pl.debian.org","name":"Poland","zone":"Europe/Warsaw"},"pm":{"kmap":"","mirror":"","name":"Saint Pierre and Miquelon","zone":"America/Miquelon"},"pn":{"kmap":"","mirror":"","name":"Pitcairn","zone":"Pacific/Pitcairn"},"pr":{"kmap":"","mirror":"","name":"Puerto Rico","zone":"America/Puerto_Rico"},"ps":{"kmap":"","mirror":"","name":"Palestine, State of","zone":"Asia/Gaza"},"pt":{"kmap":"pt","mirror":"ftp.pt.debian.org","name":"Portugal","zone":"Europe/Lisbon"},"pw":{"kmap":"","mirror":"","name":"Palau","zone":"Pacific/Palau"},"py":{"kmap":"","mirror":"","name":"Paraguay","zone":"America/Asuncion"},"qa":{"kmap":"","mirror":"","name":"Qatar","zone":"Asia/Qatar"},"re":{"kmap":"","mirror":"","name":"Réunion","zone":"Indian/Reunion"},"ro":{"kmap":"","mirror":"ftp.ro.debian.org","name":"Romania","zone":"Europe/Bucharest"},"rs":{"kmap":"","mirror":"","name":"Serbia","zone":"Europe/Belgrade"},"ru":{"kmap":"","mirror":"ftp.ru.debian.org","name":"Russian Federation","zone":"Europe/Kaliningrad"},"rw":{"kmap":"","mirror":"","name":"Rwanda","zone":"Africa/Kigali"},"sa":{"kmap":"","mirror":"","name":"Saudi Arabia","zone":"Asia/Riyadh"},"sb":{"kmap":"","mirror":"","name":"Solomon Islands","zone":"Pacific/Guadalcanal"},"sc":{"kmap":"","mirror":"","name":"Seychelles","zone":"Indian/Mahe"},"sd":{"kmap":"","mirror":"","name":"Sudan","zone":"Africa/Khartoum"},"se":{"kmap":"","mirror":"ftp.se.debian.org","name":"Sweden","zone":"Europe/Stockholm"},"sg":{"kmap":"","mirror":"","name":"Singapore","zone":"Asia/Singapore"},"sh":{"kmap":"","mirror":"","name":"Saint Helena, Ascension and Tristan da Cunha","zone":"Atlantic/St_Helena"},"si":{"kmap":"si","mirror":"ftp.si.debian.org","name":"Slovenia","zone":"Europe/Ljubljana"},"sj":{"kmap":"","mirror":"","name":"Svalbard and Jan Mayen","zone":"Arctic/Longyearbyen"},"sk":{"kmap":"","mirror":"ftp.sk.debian.org","name":"Slovakia","zone":"Europe/Bratislava"},"sl":{"kmap":"","mirror":"","name":"Sierra Leone","zone":"Africa/Freetown"},"sm":{"kmap":"","mirror":"","name":"San Marino","zone":"Europe/San_Marino"},"sn":{"kmap":"","mirror":"","name":"Senegal","zone":"Africa/Dakar"},"so":{"kmap":"","mirror":"","name":"Somalia","zone":"Africa/Mogadishu"},"sr":{"kmap":"","mirror":"","name":"Suriname","zone":"America/Paramaribo"},"ss":{"kmap":"","mirror":"","name":"South Sudan","zone":"Africa/Juba"},"st":{"kmap":"","mirror":"","name":"Sao Tome and Principe","zone":"Africa/Sao_Tome"},"sv":{"kmap":"","mirror":"","name":"El Salvador","zone":"America/El_Salvador"},"sx":{"kmap":"","mirror":"","name":"Sint Maarten (Dutch part)","zone":"America/Lower_Princes"},"sy":{"kmap":"","mirror":"","name":"Syria","zone":"Asia/Damascus"},"sz":{"kmap":"","mirror":"","name":"Eswatini","zone":"Africa/Mbabane"},"tc":{"kmap":"","mirror":"","name":"Turks and Caicos Islands","zone":"America/Grand_Turk"},"td":{"kmap":"","mirror":"","name":"Chad","zone":"Africa/Ndjamena"},"tf":{"kmap":"","mirror":"","name":"French Southern Territories","zone":"Indian/Kerguelen"},"tg":{"kmap":"","mirror":"","name":"Togo","zone":"Africa/Lome"},"th":{"kmap":"","mirror":"","name":"Thailand","zone":"Asia/Bangkok"},"tj":{"kmap":"","mirror":"","name":"Tajikistan","zone":"Asia/Dushanbe"},"tk":{"kmap":"","mirror":"","name":"Tokelau","zone":"Pacific/Fakaofo"},"tl":{"kmap":"","mirror":"","name":"Timor-Leste","zone":"Asia/Dili"},"tm":{"kmap":"","mirror":"","name":"Turkmenistan","zone":"Asia/Ashgabat"},"tn":{"kmap":"","mirror":"","name":"Tunisia","zone":"Africa/Tunis"},"to":{"kmap":"","mirror":"","name":"Tonga","zone":"Pacific/Tongatapu"},"tr":{"kmap":"","mirror":"ftp.tr.debian.org","name":"Türkiye","zone":"Europe/Istanbul"},"tt":{"kmap":"","mirror":"","name":"Trinidad and Tobago","zone":"America/Port_of_Spain"},"tv":{"kmap":"","mirror":"","name":"Tuvalu","zone":"Pacific/Funafuti"},"tw":{"kmap":"","mirror":"ftp.tw.debian.org","name":"Taiwan","zone":"Asia/Taipei"},"tz":{"kmap":"","mirror":"","name":"Tanzania","zone":"Africa/Dar_es_Salaam"},"ua":{"kmap":"","mirror":"","name":"Ukraine","zone":"Europe/Simferopol"},"ug":{"kmap":"","mirror":"","name":"Uganda","zone":"Africa/Kampala"},"um":{"kmap":"","mirror":"","name":"United States Minor Outlying Islands","zone":"Pacific/Midway"},"us":{"kmap":"en-us","mirror":"ftp.us.debian.org","name":"United States","zone":"America/New_York"},"uy":{"kmap":"","mirror":"","name":"Uruguay","zone":"America/Montevideo"},"uz":{"kmap":"","mirror":"","name":"Uzbekistan","zone":"Asia/Samarkand"},"va":{"kmap":"it","mirror":"","name":"Holy See (Vatican City State)","zone":"Europe/Vatican"},"vc":{"kmap":"","mirror":"","name":"Saint Vincent and the Grenadines","zone":"America/St_Vincent"},"ve":{"kmap":"","mirror":"","name":"Venezuela","zone":"America/Caracas"},"vg":{"kmap":"","mirror":"","name":"Virgin Islands, British","zone":"America/Tortola"},"vi":{"kmap":"","mirror":"","name":"Virgin Islands, U.S.","zone":"America/St_Thomas"},"vn":{"kmap":"","mirror":"","name":"Vietnam","zone":"Asia/Ho_Chi_Minh"},"vu":{"kmap":"","mirror":"","name":"Vanuatu","zone":"Pacific/Efate"},"wf":{"kmap":"","mirror":"","name":"Wallis and Futuna","zone":"Pacific/Wallis"},"ws":{"kmap":"","mirror":"","name":"Samoa","zone":"Pacific/Apia"},"ye":{"kmap":"","mirror":"","name":"Yemen","zone":"Asia/Aden"},"yt":{"kmap":"","mirror":"","name":"Mayotte","zone":"Indian/Mayotte"},"za":{"kmap":"","mirror":"","name":"South Africa","zone":"Africa/Johannesburg"},"zm":{"kmap":"","mirror":"","name":"Zambia","zone":"Africa/Lusaka"},"zw":{"kmap":"","mirror":"","name":"Zimbabwe","zone":"Africa/Harare"}},"countryhash":{"afghanistan":"af","albania":"al","algeria":"dz","american samoa":"as","andorra":"ad","angola":"ao","anguilla":"ai","antarctica":"aq","antigua and barbuda":"ag","argentina":"ar","armenia":"am","aruba":"aw","australia":"au","austria":"at","azerbaijan":"az","bahamas":"bs","bahrain":"bh","bangladesh":"bd","barbados":"bb","belarus":"by","belgium":"be","belize":"bz","benin":"bj","bermuda":"bm","bhutan":"bt","bolivia":"bo","bonaire, sint eustatius and saba":"bq","bosnia and herzegovina":"ba","botswana":"bw","bouvet island":"bv","brazil":"br","british indian ocean territory":"io","brunei darussalam":"bn","bulgaria":"bg","burkina faso":"bf","burundi":"bi","cabo verde":"cv","cambodia":"kh","cameroon":"cm","canada":"ca","cayman islands":"ky","central african republic":"cf","chad":"td","chile":"cl","china":"cn","christmas island":"cx","cocos (keeling) islands":"cc","colombia":"co","comoros":"km","congo":"cg","congo, the democratic republic of the":"cd","cook islands":"ck","costa rica":"cr","croatia":"hr","cuba":"cu","curaçao":"cw","cyprus":"cy","czechia":"cz","côte d'ivoire":"ci","denmark":"dk","djibouti":"dj","dominica":"dm","dominican republic":"do","ecuador":"ec","egypt":"eg","el salvador":"sv","equatorial guinea":"gq","eritrea":"er","estonia":"ee","eswatini":"sz","ethiopia":"et","falkland islands (malvinas)":"fk","faroe islands":"fo","fiji":"fj","finland":"fi","france":"fr","french guiana":"gf","french polynesia":"pf","french southern territories":"tf","gabon":"ga","gambia":"gm","georgia":"ge","germany":"de","ghana":"gh","gibraltar":"gi","greece":"gr","greenland":"gl","grenada":"gd","guadeloupe":"gp","guam":"gu","guatemala":"gt","guernsey":"gg","guinea":"gn","guinea-bissau":"gw","guyana":"gy","haiti":"ht","heard island and mcdonald islands":"hm","holy see (vatican city state)":"va","honduras":"hn","hong kong":"hk","hungary":"hu","iceland":"is","india":"in","indonesia":"id","iran":"ir","iraq":"iq","ireland":"ie","isle of man":"im","israel":"il","italy":"it","jamaica":"jm","japan":"jp","jersey":"je","jordan":"jo","kazakhstan":"kz","kenya":"ke","kiribati":"ki","kuwait":"kw","kyrgyzstan":"kg","laos":"la","latvia":"lv","lebanon":"lb","lesotho":"ls","liberia":"lr","libya":"ly","liechtenstein":"li","lithuania":"lt","luxembourg":"lu","macao":"mo","madagascar":"mg","malawi":"mw","malaysia":"my","maldives":"mv","mali":"ml","malta":"mt","marshall islands":"mh","martinique":"mq","mauritania":"mr","mauritius":"mu","mayotte":"yt","mexico":"mx","micronesia, federated states of":"fm","moldova":"md","monaco":"mc","mongolia":"mn","montenegro":"me","montserrat":"ms","morocco":"ma","mozambique":"mz","myanmar":"mm","namibia":"na","nauru":"nr","nepal":"np","netherlands":"nl","new caledonia":"nc","new zealand":"nz","nicaragua":"ni","niger":"ne","nigeria":"ng","niue":"nu","norfolk island":"nf","north korea":"kp","north macedonia":"mk","northern mariana islands":"mp","norway":"no","oman":"om","pakistan":"pk","palau":"pw","palestine, state of":"ps","panama":"pa","papua new guinea":"pg","paraguay":"py","peru":"pe","philippines":"ph","pitcairn":"pn","poland":"pl","portugal":"pt","puerto rico":"pr","qatar":"qa","romania":"ro","russian federation":"ru","rwanda":"rw","réunion":"re","saint barthélemy":"bl","saint helena, ascension and tristan da cunha":"sh","saint kitts and nevis":"kn","saint lucia":"lc","saint martin (french part)":"mf","saint pierre and miquelon":"pm","saint vincent and the grenadines":"vc","samoa":"ws","san marino":"sm","sao tome and principe":"st","saudi arabia":"sa","senegal":"sn","serbia":"rs","seychelles":"sc","sierra leone":"sl","singapore":"sg","sint maarten (dutch part)":"sx","slovakia":"sk","slovenia":"si","solomon islands":"sb","somalia":"so","south africa":"za","south georgia and the south sandwich islands":"gs","south korea":"kr","south sudan":"ss","spain":"es","sri lanka":"lk","sudan":"sd","suriname":"sr","svalbard and jan mayen":"sj","sweden":"se","switzerland":"ch","syria":"sy","taiwan":"tw","tajikistan":"tj","tanzania":"tz","thailand":"th","timor-leste":"tl","togo":"tg","tokelau":"tk","tonga":"to","trinidad and tobago":"tt","tunisia":"tn","turkmenistan":"tm","turks and caicos islands":"tc","tuvalu":"tv","türkiye":"tr","uganda":"ug","ukraine":"ua","united arab emirates":"ae","united kingdom":"gb","united states":"us","united states minor outlying islands":"um","uruguay":"uy","uzbekistan":"uz","vanuatu":"vu","venezuela":"ve","vietnam":"vn","virgin islands, british":"vg","virgin islands, u.s.":"vi","wallis and futuna":"wf","western sahara":"eh","yemen":"ye","zambia":"zm","zimbabwe":"zw","åland islands":"ax"},"kmap":{"de":{"console":"qwertz/de-latin1-nodeadkeys.kmap.gz","kvm":"de","name":"German","x11":"de","x11var":"nodeadkeys"},"de-ch":{"console":"qwertz/sg-latin1.kmap.gz","kvm":"de-ch","name":"Swiss-German","x11":"ch","x11var":"de_nodeadkeys"},"dk":{"console":"qwerty/dk-latin1.kmap.gz","kvm":"da","name":"Danish","x11":"dk","x11var":"nodeadkeys"},"en-gb":{"console":"qwerty/uk.kmap.gz","kvm":"en-gb","name":"United Kingdom","x11":"gb","x11var":""},"en-us":{"console":"qwerty/us-latin1.kmap.gz","kvm":"en-us","name":"U.S. English","x11":"us","x11var":""},"es":{"console":"qwerty/es.kmap.gz","kvm":"es","name":"Spanish","x11":"es","x11var":"nodeadkeys"},"fi":{"console":"qwerty/fi-latin1.kmap.gz","kvm":"fi","name":"Finnish","x11":"fi","x11var":"nodeadkeys"},"fr":{"console":"azerty/fr-latin1.kmap.gz","kvm":"fr","name":"French","x11":"fr","x11var":"nodeadkeys"},"fr-be":{"console":"azerty/be2-latin1.kmap.gz","kvm":"fr-be","name":"Belgium-French","x11":"be","x11var":"nodeadkeys"},"fr-ca":{"console":"qwerty/cf.kmap.gz","kvm":"fr-ca","name":"Canada-French","x11":"ca","x11var":"fr-legacy"},"fr-ch":{"console":"qwertz/fr_CH-latin1.kmap.gz","kvm":"fr-ch","name":"Swiss-French","x11":"ch","x11var":"fr_nodeadkeys"},"hu":{"console":"qwertz/hu.kmap.gz","kvm":"hu","name":"Hungarian","x11":"hu","x11var":""},"is":{"console":"qwerty/is-latin1.kmap.gz","kvm":"is","name":"Icelandic","x11":"is","x11var":"nodeadkeys"},"it":{"console":"qwerty/it2.kmap.gz","kvm":"it","name":"Italian","x11":"it","x11var":"nodeadkeys"},"jp":{"console":"qwerty/jp106.kmap.gz","kvm":"ja","name":"Japanese","x11":"jp","x11var":""},"lt":{"console":"qwerty/lt.kmap.gz","kvm":"lt","name":"Lithuanian","x11":"lt","x11var":"std"},"mk":{"console":"qwerty/mk.kmap.gz","kvm":"mk","name":"Macedonian","x11":"mk","x11var":"nodeadkeys"},"nl":{"console":"qwerty/nl.kmap.gz","kvm":"nl","name":"Dutch","x11":"nl","x11var":""},"no":{"console":"qwerty/no-latin1.kmap.gz","kvm":"no","name":"Norwegian","x11":"no","x11var":"nodeadkeys"},"pl":{"console":"qwerty/pl.kmap.gz","kvm":"pl","name":"Polish","x11":"pl","x11var":""},"pt":{"console":"qwerty/pt-latin1.kmap.gz","kvm":"pt","name":"Portuguese","x11":"pt","x11var":"nodeadkeys"},"pt-br":{"console":"qwerty/br-latin1.kmap.gz","kvm":"pt-br","name":"Brazil-Portuguese","x11":"br","x11var":"nodeadkeys"},"se":{"console":"qwerty/se-latin1.kmap.gz","kvm":"sv","name":"Swedish","x11":"se","x11var":"nodeadkeys"},"si":{"console":"qwertz/slovene.kmap.gz","kvm":"sl","name":"Slovenian","x11":"si","x11var":""},"tr":{"console":"qwerty/trq.kmap.gz","kvm":"tr","name":"Turkish","x11":"tr","x11var":""}},"kmaphash":{"Belgium-French":"fr-be","Brazil-Portuguese":"pt-br","Canada-French":"fr-ca","Danish":"dk","Dutch":"nl","Finnish":"fi","French":"fr","German":"de","Hungarian":"hu","Icelandic":"is","Italian":"it","Japanese":"jp","Lithuanian":"lt","Macedonian":"mk","Norwegian":"no","Polish":"pl","Portuguese":"pt","Slovenian":"si","Spanish":"es","Swedish":"se","Swiss-French":"fr-ch","Swiss-German":"de-ch","Turkish":"tr","U.S. English":"en-us","United Kingdom":"en-gb"},"zones":{"Africa/Abidjan":1,"Africa/Accra":1,"Africa/Addis_Ababa":1,"Africa/Algiers":1,"Africa/Asmara":1,"Africa/Bamako":1,"Africa/Bangui":1,"Africa/Banjul":1,"Africa/Bissau":1,"Africa/Blantyre":1,"Africa/Brazzaville":1,"Africa/Bujumbura":1,"Africa/Cairo":1,"Africa/Casablanca":1,"Africa/Ceuta":1,"Africa/Conakry":1,"Africa/Dakar":1,"Africa/Dar_es_Salaam":1,"Africa/Djibouti":1,"Africa/Douala":1,"Africa/El_Aaiun":1,"Africa/Freetown":1,"Africa/Gaborone":1,"Africa/Harare":1,"Africa/Johannesburg":1,"Africa/Juba":1,"Africa/Kampala":1,"Africa/Khartoum":1,"Africa/Kigali":1,"Africa/Kinshasa":1,"Africa/Lagos":1,"Africa/Libreville":1,"Africa/Lome":1,"Africa/Luanda":1,"Africa/Lubumbashi":1,"Africa/Lusaka":1,"Africa/Malabo":1,"Africa/Maputo":1,"Africa/Maseru":1,"Africa/Mbabane":1,"Africa/Mogadishu":1,"Africa/Monrovia":1,"Africa/Nairobi":1,"Africa/Ndjamena":1,"Africa/Niamey":1,"Africa/Nouakchott":1,"Africa/Ouagadougou":1,"Africa/Porto-Novo":1,"Africa/Sao_Tome":1,"Africa/Tripoli":1,"Africa/Tunis":1,"Africa/Windhoek":1,"America/Adak":1,"America/Anchorage":1,"America/Anguilla":1,"America/Antigua":1,"America/Araguaina":1,"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1,"America/Aruba":1,"America/Asuncion":1,"America/Atikokan":1,"America/Bahia":1,"America/Bahia_Banderas":1,"America/Barbados":1,"America/Belem":1,"America/Belize":1,"America/Blanc-Sablon":1,"America/Boa_Vista":1,"America/Bogota":1,"America/Boise":1,"America/Cambridge_Bay":1,"America/Campo_Grande":1,"America/Cancun":1,"America/Caracas":1,"America/Cayenne":1,"America/Cayman":1,"America/Chicago":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Costa_Rica":1,"America/Creston":1,"America/Cuiaba":1,"America/Curacao":1,"America/Danmarkshavn":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Denver":1,"America/Detroit":1,"America/Dominica":1,"America/Edmonton":1,"America/Eirunepe":1,"America/El_Salvador":1,"America/Fort_Nelson":1,"America/Fortaleza":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Grand_Turk":1,"America/Grenada":1,"America/Guadeloupe":1,"America/Guatemala":1,"America/Guayaquil":1,"America/Guyana":1,"America/Halifax":1,"America/Havana":1,"America/Hermosillo":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Jamaica":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Kralendijk":1,"America/La_Paz":1,"America/Lima":1,"America/Los_Angeles":1,"America/Lower_Princes":1,"America/Maceio":1,"America/Managua":1,"America/Manaus":1,"America/Marigot":1,"America/Martinique":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Menominee":1,"America/Merida":1,"America/Metlakatla":1,"America/Mexico_City":1,"America/Miquelon":1,"America/Moncton":1,"America/Monterrey":1,"America/Montevideo":1,"America/Montserrat":1,"America/Nassau":1,"America/New_York":1,"America/Nome":1,"America/Noronha":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Nuuk":1,"America/Ojinaga":1,"America/Panama":1,"America/Paramaribo":1,"America/Phoenix":1,"America/Port-au-Prince":1,"America/Port_of_Spain":1,"America/Porto_Velho":1,"America/Puerto_Rico":1,"America/Punta_Arenas":1,"America/Rankin_Inlet":1,"America/Recife":1,"America/Regina":1,"America/Resolute":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Santiago":1,"America/Santo_Domingo":1,"America/Sao_Paulo":1,"America/Scoresbysund":1,"America/Sitka":1,"America/St_Barthelemy":1,"America/St_Johns":1,"America/St_Kitts":1,"America/St_Lucia":1,"America/St_Thomas":1,"America/St_Vincent":1,"America/Swift_Current":1,"America/Tegucigalpa":1,"America/Thule":1,"America/Tijuana":1,"America/Toronto":1,"America/Tortola":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1,"America/Yakutat":1,"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Macquarie":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1,"Arctic/Longyearbyen":1,"Asia/Aden":1,"Asia/Almaty":1,"Asia/Amman":1,"Asia/Anadyr":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Ashgabat":1,"Asia/Atyrau":1,"Asia/Baghdad":1,"Asia/Bahrain":1,"Asia/Baku":1,"Asia/Bangkok":1,"Asia/Barnaul":1,"Asia/Beirut":1,"Asia/Bishkek":1,"Asia/Brunei":1,"Asia/Chita":1,"Asia/Choibalsan":1,"Asia/Colombo":1,"Asia/Damascus":1,"Asia/Dhaka":1,"Asia/Dili":1,"Asia/Dubai":1,"Asia/Dushanbe":1,"Asia/Famagusta":1,"Asia/Gaza":1,"Asia/Hebron":1,"Asia/Ho_Chi_Minh":1,"Asia/Hong_Kong":1,"Asia/Hovd":1,"Asia/Irkutsk":1,"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Jerusalem":1,"Asia/Kabul":1,"Asia/Kamchatka":1,"Asia/Karachi":1,"Asia/Kathmandu":1,"Asia/Khandyga":1,"Asia/Kolkata":1,"Asia/Krasnoyarsk":1,"Asia/Kuala_Lumpur":1,"Asia/Kuching":1,"Asia/Kuwait":1,"Asia/Macau":1,"Asia/Magadan":1,"Asia/Makassar":1,"Asia/Manila":1,"Asia/Muscat":1,"Asia/Nicosia":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Oral":1,"Asia/Phnom_Penh":1,"Asia/Pontianak":1,"Asia/Pyongyang":1,"Asia/Qatar":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1,"Asia/Riyadh":1,"Asia/Sakhalin":1,"Asia/Samarkand":1,"Asia/Seoul":1,"Asia/Shanghai":1,"Asia/Singapore":1,"Asia/Srednekolymsk":1,"Asia/Taipei":1,"Asia/Tashkent":1,"Asia/Tbilisi":1,"Asia/Tehran":1,"Asia/Thimphu":1,"Asia/Tokyo":1,"Asia/Tomsk":1,"Asia/Ulaanbaatar":1,"Asia/Urumqi":1,"Asia/Ust-Nera":1,"Asia/Vientiane":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yangon":1,"Asia/Yekaterinburg":1,"Asia/Yerevan":1,"Atlantic/Azores":1,"Atlantic/Bermuda":1,"Atlantic/Canary":1,"Atlantic/Cape_Verde":1,"Atlantic/Faroe":1,"Atlantic/Madeira":1,"Atlantic/Reykjavik":1,"Atlantic/South_Georgia":1,"Atlantic/St_Helena":1,"Atlantic/Stanley":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1,"Europe/Amsterdam":1,"Europe/Andorra":1,"Europe/Astrakhan":1,"Europe/Athens":1,"Europe/Belgrade":1,"Europe/Berlin":1,"Europe/Bratislava":1,"Europe/Brussels":1,"Europe/Bucharest":1,"Europe/Budapest":1,"Europe/Busingen":1,"Europe/Chisinau":1,"Europe/Copenhagen":1,"Europe/Dublin":1,"Europe/Gibraltar":1,"Europe/Guernsey":1,"Europe/Helsinki":1,"Europe/Isle_of_Man":1,"Europe/Istanbul":1,"Europe/Jersey":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Kyiv":1,"Europe/Lisbon":1,"Europe/Ljubljana":1,"Europe/London":1,"Europe/Luxembourg":1,"Europe/Madrid":1,"Europe/Malta":1,"Europe/Mariehamn":1,"Europe/Minsk":1,"Europe/Monaco":1,"Europe/Moscow":1,"Europe/Oslo":1,"Europe/Paris":1,"Europe/Podgorica":1,"Europe/Prague":1,"Europe/Riga":1,"Europe/Rome":1,"Europe/Samara":1,"Europe/San_Marino":1,"Europe/Sarajevo":1,"Europe/Saratov":1,"Europe/Simferopol":1,"Europe/Skopje":1,"Europe/Sofia":1,"Europe/Stockholm":1,"Europe/Tallinn":1,"Europe/Tirane":1,"Europe/Ulyanovsk":1,"Europe/Vaduz":1,"Europe/Vatican":1,"Europe/Vienna":1,"Europe/Vilnius":1,"Europe/Volgograd":1,"Europe/Warsaw":1,"Europe/Zagreb":1,"Europe/Zurich":1,"Indian/Antananarivo":1,"Indian/Chagos":1,"Indian/Christmas":1,"Indian/Cocos":1,"Indian/Comoro":1,"Indian/Kerguelen":1,"Indian/Mahe":1,"Indian/Maldives":1,"Indian/Mauritius":1,"Indian/Mayotte":1,"Indian/Reunion":1,"Pacific/Apia":1,"Pacific/Auckland":1,"Pacific/Bougainville":1,"Pacific/Chatham":1,"Pacific/Chuuk":1,"Pacific/Easter":1,"Pacific/Efate":1,"Pacific/Fakaofo":1,"Pacific/Fiji":1,"Pacific/Funafuti":1,"Pacific/Galapagos":1,"Pacific/Gambier":1,"Pacific/Guadalcanal":1,"Pacific/Guam":1,"Pacific/Honolulu":1,"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Kosrae":1,"Pacific/Kwajalein":1,"Pacific/Majuro":1,"Pacific/Marquesas":1,"Pacific/Midway":1,"Pacific/Nauru":1,"Pacific/Niue":1,"Pacific/Norfolk":1,"Pacific/Noumea":1,"Pacific/Pago_Pago":1,"Pacific/Palau":1,"Pacific/Pitcairn":1,"Pacific/Pohnpei":1,"Pacific/Port_Moresby":1,"Pacific/Rarotonga":1,"Pacific/Saipan":1,"Pacific/Tahiti":1,"Pacific/Tarawa":1,"Pacific/Tongatapu":1,"Pacific/Wake":1,"Pacific/Wallis":1}}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
new file mode 100644
index 0000000..f228b1e
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
@@ -0,0 +1,29 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "6": "6",
+ "7": "7",
+ "8": "8",
+ "9": "9"
+ },
+ "filesys": "zfs (RAID10)",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
new file mode 100644
index 0000000..458351e
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid10"
+#disk_list = ['sda']
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
new file mode 100644
index 0000000..2990922
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
@@ -0,0 +1,26 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "9": "9"
+ },
+ "filesys": "zfs (RAID0)",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
new file mode 100644
index 0000000..ed05f89
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid0"
+filter_match = "all"
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+filter.ID_SERIAL_SHORT = "S2HRNX0J403419"
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
new file mode 100644
index 0000000..8c22300
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
@@ -0,0 +1,33 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "0": "0",
+ "1": "1",
+ "2": "2",
+ "3": "3",
+ "6": "6",
+ "7": "7",
+ "8": "8",
+ "9": "9"
+ },
+ "filesys": "zfs (RAID10)",
+ "gateway": "192.168.1.1",
+ "hdsize": 2980.820640563965,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
new file mode 100644
index 0000000..420fb40
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid10"
+filter_match = "any"
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+filter.ID_MODEL = "Micron_9300*"
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.json b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json
new file mode 100644
index 0000000..9021377
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
new file mode 100644
index 0000000..4900ca5
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
@@ -0,0 +1,14 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
new file mode 100644
index 0000000..6f31079
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "10.10.10.10/24",
+ "country": "at",
+ "dns": "10.10.10.1",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "10.10.10.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "enp65s0f0",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
new file mode 100644
index 0000000..37e8a22
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
@@ -0,0 +1,19 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = false
+cidr = "10.10.10.10/24"
+dns = "10.10.10.1"
+gateway = "10.10.10.1"
+filter.ID_NET_NAME_MAC = "*a0369f0ab382"
+
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/readme b/proxmox-auto-installer/tests/resources/parse_answer/readme
new file mode 100644
index 0000000..6ce77ae
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/readme
@@ -0,0 +1,4 @@
+the size parameter from /sys/block/{disk}/size is the number of blocks.
+
+to calculate the size as the low level installer needs it:
+size * 512 / 1024 / 1024 / 1024 with 14 digits after the comma
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
new file mode 100644
index 0000000..515cc89
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "10.10.10.10/24",
+ "country": "at",
+ "dns": "10.10.10.1",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "10.10.10.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "enp129s0f1np1",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
new file mode 100644
index 0000000..b7d7728
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
@@ -0,0 +1,19 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = false
+cidr = "10.10.10.10/24"
+dns = "10.10.10.1"
+gateway = "10.10.10.1"
+filter.ID_NET_NAME = "enp129s0f1np1"
+
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.json b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json
new file mode 100644
index 0000000..c29a303
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json
@@ -0,0 +1,27 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "6": "6",
+ "7": "7"
+ },
+ "filesys": "zfs (RAID1)",
+ "gateway": "192.168.1.1",
+ "hdsize": 80.0,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "lz4",
+ "copies": 2
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
new file mode 100644
index 0000000..5410ed6
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
@@ -0,0 +1,20 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid1"
+zfs.ashift = 12
+zfs.checksum = "on"
+zfs.compress = "lz4"
+zfs.copies = 2
+zfs.hdsize = 80
+disk_list = ["sda", "sdb"]
diff --git a/proxmox-auto-installer/tests/resources/run-env-info.json b/proxmox-auto-installer/tests/resources/run-env-info.json
new file mode 100644
index 0000000..6762470
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/run-env-info.json
@@ -0,0 +1 @@
+{"boot_type":"efi","country":"at","disks":[[0,"/dev/nvme0n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme0n1"],[1,"/dev/nvme1n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme1n1"],[2,"/dev/nvme2n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme2n1"],[3,"/dev/nvme3n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme3n1"],[4,"/dev/nvme4n1",976773168,"Samsung SSD 970 EVO Plus 500GB",512,"/sys/block/nvme4n1"],[5,"/dev/nvme5n1",732585168,"INTEL SSDPED1K375GA",512,"/sys/block/nvme5n1"],[6,"/dev/sda",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sda"],[7,"/dev/sdb",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdb"],[8,"/dev/sdc",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdc"],[9,"/dev/sdd",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdd"]],"hvm_supported":1,"ipconf":{"default":"4","dnsserver":"192.168.1.254","domain":null,"gateway":"192.168.1.1","ifaces":{"10":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"2":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"3":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"4":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","inet":{"addr":"192.168.1.114","mask":"255.255.240.0","prefix":20},"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"5":{"driver":"cdc_ether","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"},"6":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"7":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"8":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"9":{"driver":"mlx5_core","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"}}},"kernel_cmdline":"BOOT_IMAGE=/boot/linux26 ro ramdisk_size=16777216 rw splash=verbose proxdebug vga=788","network":{"dns":{"dns":["192.168.1.254"],"domain":null},"interfaces":{"eno1":{"addresses":[{"address":"192.168.1.114","family":"inet","prefix":24}],"index":4,"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"eno2":{"index":6,"mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"enp129s0f0np0":{"index":7,"mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"enp129s0f1np1":{"index":8,"mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"enp193s0f0np0":{"index":9,"mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"},"enp193s0f1np1":{"index":10,"mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"enp65s0f0":{"index":2,"mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"enp65s0f1":{"index":3,"mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"enx5a4732ddc747":{"index":5,"mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"}},"routes":{"gateway4":{"dev":"eno1","gateway":"192.168.1.1"}}},"total_memory":257597}
diff --git a/proxmox-auto-installer/tests/resources/run-env-udev.json b/proxmox-auto-installer/tests/resources/run-env-udev.json
new file mode 100644
index 0000000..4fe1f30
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/run-env-udev.json
@@ -0,0 +1 @@
+{"disks":{"0":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:01:00.0-nvme-1 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_19502596FC74 /dev/disk/by-id/lvm-pv-uuid-hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP /dev/disk/by-diskseq/16 /dev/disk/by-id/nvme-eui.000000000000001500a075012596fc74","DEVNAME":"/dev/nvme0n1","DEVPATH":"/devices/pci0000:00/0000:00:01.1/0000:01:00.0/nvme/nvme0/nvme0n1","DEVTYPE":"disk","DISKSEQ":"16","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_UUID_ENC":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:01:00.0-nvme-1","ID_PATH_TAG":"pci-0000_01_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_19502596FC74","ID_SERIAL_SHORT":"19502596FC74","ID_WWN":"eui.000000000000001500a075012596fc74","LVM_VG_NAME_COMPLETE":"ceph-67f6a633-8bac-4ba6-a54c-40f0d24a9701","MAJOR":"259","MINOR":"6","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215609"},"1":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:02:00.0-nvme-1 /dev/disk/by-id/nvme-eui.000000000000001400a0750125de7a16 /dev/disk/by-diskseq/15 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_195225DE7A16","DEVNAME":"/dev/nvme1n1","DEVPATH":"/devices/pci0000:00/0000:00:01.2/0000:02:00.0/nvme/nvme1/nvme1n1","DEVTYPE":"disk","DISKSEQ":"15","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:02:00.0-nvme-1","ID_PATH_TAG":"pci-0000_02_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_195225DE7A16","ID_SERIAL_SHORT":"195225DE7A16","ID_WWN":"eui.000000000000001400a0750125de7a16","MAJOR":"259","MINOR":"5","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271971"},"2":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:03:00.0-nvme-1 /dev/disk/by-diskseq/17 /dev/disk/by-id/lvm-pv-uuid-b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F206E /dev/disk/by-id/nvme-eui.000000000000001400a07501250f206e","DEVNAME":"/dev/nvme2n1","DEVPATH":"/devices/pci0000:00/0000:00:01.3/0000:03:00.0/nvme/nvme2/nvme2n1","DEVTYPE":"disk","DISKSEQ":"17","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_UUID_ENC":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:03:00.0-nvme-1","ID_PATH_TAG":"pci-0000_03_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F206E","ID_SERIAL_SHORT":"1945250F206E","ID_WWN":"eui.000000000000001400a07501250f206e","LVM_VG_NAME_COMPLETE":"ceph-ee820014-6121-458b-a661-889f0901bff6","MAJOR":"259","MINOR":"7","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45218640"},"3":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:04:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F20AC /dev/disk/by-diskseq/18 /dev/disk/by-id/nvme-eui.000000000000001400a07501250f20ac","DEVNAME":"/dev/nvme3n1","DEVPATH":"/devices/pci0000:00/0000:00:01.4/0000:04:00.0/nvme/nvme3/nvme3n1","DEVTYPE":"disk","DISKSEQ":"18","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_UUID_ENC":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:04:00.0-nvme-1","ID_PATH_TAG":"pci-0000_04_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F20AC","ID_SERIAL_SHORT":"1945250F20AC","ID_WWN":"eui.000000000000001400a07501250f20ac","LVM_VG_NAME_COMPLETE":"ceph-2928aceb-9300-4175-8640-e227d897d45e","MAJOR":"259","MINOR":"8","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215244"},"4":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/13 /dev/disk/by-path/pci-0000:82:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd /dev/disk/by-id/nvme-INTEL_SSDPED1K375GA_PHKS746500DK375AGN /dev/disk/by-id/nvme-nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","DEVNAME":"/dev/nvme4n1","DEVPATH":"/devices/pci0000:80/0000:80:03.1/0000:82:00.0/nvme/nvme4/nvme4n1","DEVTYPE":"disk","DISKSEQ":"13","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_UUID_ENC":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_VERSION":"LVM2 001","ID_MODEL":"INTEL SSDPED1K375GA","ID_PATH":"pci-0000:82:00.0-nvme-1","ID_PATH_TAG":"pci-0000_82_00_0-nvme-1","ID_REVISION":"E2010435","ID_SERIAL":"INTEL_SSDPED1K375GA_PHKS746500DK375AGN","ID_SERIAL_SHORT":"PHKS746500DK375AGN","ID_WWN":"nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","LVM_VG_NAME_COMPLETE":"ceph-b4af8112-88e7-4cd4-9cf9-0f4163ca77bd","MAJOR":"259","MINOR":"0","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45219471"},"5":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/nvme-eui.0025385791b04175 /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N /dev/disk/by-path/pci-0000:06:00.0-nvme-1 /dev/disk/by-diskseq/14","DEVNAME":"/dev/nvme5n1","DEVPATH":"/devices/pci0000:00/0000:00:03.3/0000:06:00.0/nvme/nvme5/nvme5n1","DEVTYPE":"disk","DISKSEQ":"14","ID_MODEL":"Samsung SSD 970 EVO Plus 500GB","ID_PART_TABLE_TYPE":"gpt","ID_PART_TABLE_UUID":"1c40cb4b-72d8-49ec-804b-e5933e09423d","ID_PATH":"pci-0000:06:00.0-nvme-1","ID_PATH_TAG":"pci-0000_06_00_0-nvme-1","ID_REVISION":"2B2QEXM7","ID_SERIAL":"Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N","ID_SERIAL_SHORT":"S4EVNF0M703256N","ID_WWN":"eui.0025385791b04175","MAJOR":"259","MINOR":"1","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271933"},"6":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/lvm-pv-uuid-tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0 /dev/disk/by-diskseq/9 /dev/disk/by-id/wwn-0x5002538c405dbf10","DEVNAME":"/dev/sda","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:0/end_device-8:0:0/target8:0:0/8:0:0:0/block/sda","DEVTYPE":"disk","DISKSEQ":"9","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_UUID_ENC":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550","ID_SERIAL_SHORT":"S2HRNX0J403550","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbf10","ID_WWN_WITH_EXTENSION":"0x5002538c405dbf10","MAJOR":"8","MINOR":"0","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45234812"},"7":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbce5 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0 /dev/disk/by-id/lvm-pv-uuid-oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu /dev/disk/by-diskseq/10 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","DEVNAME":"/dev/sdb","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:1/end_device-8:0:1/target8:0:1/8:0:1:0/block/sdb","DEVTYPE":"disk","DISKSEQ":"10","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_UUID_ENC":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","ID_SERIAL_SHORT":"S2HRNX0J403335","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbce5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbce5","MAJOR":"8","MINOR":"16","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215406"},"8":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbcd9 /dev/disk/by-diskseq/11 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0 /dev/disk/by-id/lvm-pv-uuid-tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","DEVNAME":"/dev/sdc","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:2/end_device-8:0:2/target8:0:2/8:0:2:0/block/sdc","DEVTYPE":"disk","DISKSEQ":"11","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_UUID_ENC":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333","ID_SERIAL_SHORT":"S2HRNX0J403333","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbcd9","ID_WWN_WITH_EXTENSION":"0x5002538c405dbcd9","MAJOR":"8","MINOR":"32","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45198824"},"9":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/12 /dev/disk/by-id/wwn-0x5002538c405dbdc5 /dev/disk/by-id/lvm-pv-uuid-Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","DEVNAME":"/dev/sdd","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:3/end_device-8:0:3/target8:0:3/8:0:3:0/block/sdd","DEVTYPE":"disk","DISKSEQ":"12","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_UUID_ENC":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","ID_SERIAL_SHORT":"S2HRNX0J403419","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbdc5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbdc5","MAJOR":"8","MINOR":"48","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215283"}},"nics":{"eno1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.0/net/eno1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN1","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno1","ID_NET_NAME_MAC":"enxb42e99acadb4","ID_NET_NAME_ONBOARD":"eno1","ID_NET_NAME_PATH":"enp194s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.0","ID_PATH_TAG":"pci-0000_c2_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"5","INTERFACE":"eno1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno1","TAGS":":systemd:","USEC_INITIALIZED":"45212091"},"eno2":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.1/net/eno2","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN2","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno2","ID_NET_NAME_MAC":"enxb42e99acadb5","ID_NET_NAME_ONBOARD":"eno2","ID_NET_NAME_PATH":"enp194s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.1","ID_PATH_TAG":"pci-0000_c2_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"6","INTERFACE":"eno2","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno2","TAGS":":systemd:","USEC_INITIALIZED":"45128159"},"enp129s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.0/net/enp129s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f0np0","ID_NET_NAME_MAC":"enx1c34da5c5e24","ID_NET_NAME_PATH":"enp129s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.0","ID_PATH_TAG":"pci-0000_81_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"7","INTERFACE":"enp129s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47752091"},"enp129s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.1/net/enp129s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f1np1","ID_NET_NAME_MAC":"enx1c34da5c5e25","ID_NET_NAME_PATH":"enp129s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.1","ID_PATH_TAG":"pci-0000_81_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"8","INTERFACE":"enp129s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47716100"},"enp193s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.0/net/enp193s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f0np0","ID_NET_NAME_MAC":"enx248a071e05bc","ID_NET_NAME_PATH":"enp193s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.0","ID_PATH_TAG":"pci-0000_c1_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"9","INTERFACE":"enp193s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47784094"},"enp193s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.1/net/enp193s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f1np1","ID_NET_NAME_MAC":"enx248a071e05bd","ID_NET_NAME_PATH":"enp193s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.1","ID_PATH_TAG":"pci-0000_c1_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"10","INTERFACE":"enp193s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47820155"},"enp65s0f0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.0/net/enp65s0f0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f0","ID_NET_NAME_MAC":"enxa0369f0ab382","ID_NET_NAME_PATH":"enp65s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.0","ID_PATH_TAG":"pci-0000_41_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"3","INTERFACE":"enp65s0f0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f0","TAGS":":systemd:","USEC_INITIALIZED":"45176103"},"enp65s0f1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.1/net/enp65s0f1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f1","ID_NET_NAME_MAC":"enxa0369f0ab383","ID_NET_NAME_PATH":"enp65s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.1","ID_PATH_TAG":"pci-0000_41_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"4","INTERFACE":"enp65s0f1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f1","TAGS":":systemd:","USEC_INITIALIZED":"45260218"},"enxaa0c304b6362":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:08.1/0000:43:00.3/usb3/3-2/3-2.4/3-2.4.3/3-2.4.3:2.0/net/enxaa0c304b6362","ID_BUS":"usb","ID_MODEL":"Virtual_Ethernet","ID_MODEL_ENC":"Virtual\\x20Ethernet","ID_MODEL_ID":"ffb0","ID_NET_DRIVER":"cdc_ether","ID_NET_LINK_FILE":"/usr/lib/systemd/network/73-usb-net-by-mac.link","ID_NET_NAME":"enxaa0c304b6362","ID_NET_NAME_MAC":"enxaa0c304b6362","ID_NET_NAME_PATH":"enp67s0f3u2u4u3c2","ID_NET_NAMING_SCHEME":"v252","ID_PATH":"pci-0000:43:00.3-usb-0:2.4.3:2.0","ID_PATH_TAG":"pci-0000_43_00_3-usb-0_2_4_3_2_0","ID_REVISION":"0100","ID_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_SERIAL_SHORT":"1234567890","ID_TYPE":"generic","ID_USB_CLASS_FROM_DATABASE":"Communications","ID_USB_DRIVER":"cdc_ether","ID_USB_INTERFACES":":0202ff:0a0000:020600:","ID_USB_INTERFACE_NUM":"00","ID_USB_MODEL":"Virtual_Ethernet","ID_USB_MODEL_ENC":"Virtual\\x20Ethernet","ID_USB_MODEL_ID":"ffb0","ID_USB_REVISION":"0100","ID_USB_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_USB_SERIAL_SHORT":"1234567890","ID_USB_TYPE":"generic","ID_USB_VENDOR":"American_Megatrends_Inc.","ID_USB_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_USB_VENDOR_ID":"046b","ID_VENDOR":"American_Megatrends_Inc.","ID_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_VENDOR_FROM_DATABASE":"American Megatrends, Inc.","ID_VENDOR_ID":"046b","IFINDEX":"2","INTERFACE":"enxaa0c304b6362","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enxaa0c304b6362","TAGS":":systemd:","USEC_INITIALIZED":"44748106"}}}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 14/36] auto-installer: add auto-installer binary
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (12 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 13/36] auto-installer: add tests for answer file parsing Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 15/36] auto-installer: add fetch answer binary Aaron Lauterer
` (21 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
It expects the contents of an answer file via stdin. It will then be
parsed and the JSON for the low level installer is generated.
It then calls the low level installer directly.
The output of the installaton progress is kept rather simple for now.
If configured in the answer file, commands will be run pre and post the
low level installer.
It also logs everything to the logfile, currently
'/tmp/auto_installer.log'.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 9 +-
.../src/bin/proxmox-auto-installer.rs | 195 ++++++++++++++++++
2 files changed, 201 insertions(+), 3 deletions(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
diff --git a/Makefile b/Makefile
index 4f140ca..3ac5769 100644
--- a/Makefile
+++ b/Makefile
@@ -18,7 +18,9 @@ INSTALLER_SOURCES=$(shell git ls-files) country.dat
PREFIX = /usr
BINDIR = $(PREFIX)/bin
-USR_BIN := proxmox-tui-installer
+USR_BIN := \
+ proxmox-tui-installer\
+ proxmox-auto-installer
COMPILED_BINS := \
$(addprefix $(CARGO_COMPILEDIR)/,$(USR_BIN))
@@ -99,7 +101,7 @@ VARLIBDIR=$(DESTDIR)/var/lib/proxmox-installer
HTMLDIR=$(VARLIBDIR)/html/common
.PHONY: install
-install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer
+install: $(INSTALLER_SOURCES) $(COMPILED_BINS)
$(MAKE) -C banner install
$(MAKE) -C Proxmox install
install -D -m 644 interfaces $(DESTDIR)/etc/network/interfaces
@@ -118,7 +120,8 @@ install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer
$(COMPILED_BINS): cargo-build
.PHONY: cargo-build
cargo-build:
- $(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer $(CARGO_BUILD_ARGS)
+ $(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
+ --package proxmox-auto-installer --bin proxmox-auto-installer $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
new file mode 100644
index 0000000..f43b12f
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
@@ -0,0 +1,195 @@
+use anyhow::{anyhow, bail, Error, Result};
+use log::{error, info, LevelFilter};
+use std::{
+ env,
+ io::{BufRead, BufReader, Write},
+ path::PathBuf,
+ process::ExitCode,
+};
+
+use proxmox_installer_common::setup::{
+ installer_setup, read_json, spawn_low_level_installer, LocaleInfo, RuntimeInfo, SetupInfo,
+};
+
+use proxmox_auto_installer::{
+ answer::Answer,
+ log::AutoInstLogger,
+ udevinfo::UdevInfo,
+ utils,
+ utils::{parse_answer, LowLevelMessage},
+};
+
+static LOGGER: AutoInstLogger = AutoInstLogger;
+
+pub fn init_log() -> Result<()> {
+ AutoInstLogger::init("/tmp/auto_installer.log")?;
+ log::set_logger(&LOGGER)
+ .map(|()| log::set_max_level(LevelFilter::Info))
+ .map_err(|err| anyhow!(err))
+}
+
+fn auto_installer_setup(in_test_mode: bool) -> Result<(Answer, UdevInfo)> {
+ let base_path = if in_test_mode { "./testdir" } else { "/" };
+ let mut path = PathBuf::from(base_path);
+
+ path.push("run");
+ path.push("proxmox-installer");
+
+ let udev_info: UdevInfo = {
+ let mut path = path.clone();
+ path.push("run-env-udev.json");
+
+ read_json(&path).map_err(|err| anyhow!("Failed to retrieve udev info details: {err}"))?
+ };
+
+ let mut buffer = String::new();
+ let lines = std::io::stdin().lock().lines();
+ for line in lines {
+ buffer.push_str(&line.unwrap());
+ buffer.push('\n');
+ }
+
+ let answer: Answer =
+ toml::from_str(&buffer).map_err(|err| anyhow!("Failed parsing answer file: {err}"))?;
+
+ Ok((answer, udev_info))
+}
+
+fn main() -> ExitCode {
+ if let Err(err) = init_log() {
+ panic!("could not initilize logging: {}", err);
+ }
+
+ let in_test_mode = match env::args().nth(1).as_deref() {
+ Some("-t") => true,
+ // Always force the test directory in debug builds
+ _ => cfg!(debug_assertions),
+ };
+ info!("Starting auto installer");
+
+ let (setup_info, locales, runtime_info) = match installer_setup(in_test_mode) {
+ Ok(result) => result,
+ Err(err) => {
+ error!("Installer setup error: {err}");
+ return ExitCode::FAILURE;
+ }
+ };
+
+ let (answer, udevadm_info) = match auto_installer_setup(in_test_mode) {
+ Ok(result) => result,
+ Err(err) => {
+ error!("Autoinstaller setup error: {err}");
+ return ExitCode::FAILURE;
+ }
+ };
+
+ match utils::run_cmds("Pre", &answer.global.pre_commands) {
+ Ok(_) => (),
+ Err(err) => {
+ error!("Error when running Pre-Commands: {}", err);
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ };
+ match run_installation(&answer, &locales, &runtime_info, &udevadm_info, &setup_info) {
+ Ok(_) => info!("Installation done."),
+ Err(err) => {
+ error!("Installation failed: {err}");
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ }
+ match utils::run_cmds("Post", &answer.global.post_commands) {
+ Ok(_) => (),
+ Err(err) => {
+ error!("Error when running Post-Commands: {}", err);
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ };
+ ExitCode::SUCCESS
+}
+
+/// When we exit with a failure, the installer will not automatically reboot.
+/// Default value for reboot_on_error is false
+fn exit_failure(reboot_on_error: bool) -> ExitCode {
+ if reboot_on_error {
+ ExitCode::SUCCESS
+ } else {
+ ExitCode::FAILURE
+ }
+}
+
+fn run_installation(
+ answer: &Answer,
+ locales: &LocaleInfo,
+ runtime_info: &RuntimeInfo,
+ udevadm_info: &UdevInfo,
+ setup_info: &SetupInfo,
+) -> Result<()> {
+ let config = parse_answer(answer, udevadm_info, runtime_info, locales, setup_info)?;
+ info!("Calling low-level installer");
+
+ let mut child = match spawn_low_level_installer(false) {
+ Ok(child) => child,
+ Err(err) => {
+ bail!("Low level installer could not be started: {}", err);
+ }
+ };
+
+ let mut cur_counter = 111;
+ let mut inner = || -> Result<()> {
+ let reader = child
+ .stdout
+ .take()
+ .map(BufReader::new)
+ .ok_or(anyhow!("failed to get stdout reader"))?;
+ let mut writer = child
+ .stdin
+ .take()
+ .ok_or(anyhow!("failed to get stdin writer"))?;
+
+ serde_json::to_writer(&mut writer, &config)
+ .map_err(|err| anyhow!("failed to serialize install config: {err}"))?;
+ writeln!(writer).map_err(|err| anyhow!("failed to write install config: {err}"))?;
+
+ for line in reader.lines() {
+ let line = match line {
+ Ok(line) => line,
+ Err(_) => break,
+ };
+ let msg = match serde_json::from_str::<LowLevelMessage>(&line) {
+ Ok(msg) => msg,
+ Err(_) => {
+ // Not a fatal error, so don't abort the installation by returning
+ continue;
+ }
+ };
+
+ match msg.clone() {
+ LowLevelMessage::Info { message } => info!("{message}"),
+ LowLevelMessage::Error { message } => error!("{message}"),
+ LowLevelMessage::Prompt { query } => {
+ bail!("Got interactive prompt I cannot answer: {query}")
+ }
+ LowLevelMessage::Progress { ratio, text: _ } => {
+ let counter = (ratio * 100.).floor() as usize;
+ if counter != cur_counter {
+ cur_counter = counter;
+ info!("Progress: {counter:>3}%");
+ }
+ }
+ LowLevelMessage::Finished { state, message } => {
+ if state == "err" {
+ bail!("{message}");
+ }
+ info!("Finished: '{state}' {message}");
+ }
+ };
+ }
+ Ok(())
+ };
+ match inner() {
+ Err(err) => Err(Error::msg(format!(
+ "low level installer returned early: {err}"
+ ))),
+ _ => Ok(()),
+ }
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 15/36] auto-installer: add fetch answer binary
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (13 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 14/36] auto-installer: add auto-installer binary Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 16/36] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
` (20 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
it is supposed to be run first and fetch an answer file.
The initial implementation searches for a partition/filesystem called
'proxmoxinst' or 'PROXMOXINST' with an 'answer.toml' file in the root
directory.
Once it has an answer file, it will call the 'proxmox-auto-installer'
and pipe in the contents via stdin.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 4 +-
.../src/bin/proxmox-fetch-answer.rs | 71 ++++++++++++++++
.../src/fetch_plugins/mod.rs | 2 +
.../src/fetch_plugins/partition.rs | 32 ++++++++
.../src/fetch_plugins/utils.rs | 81 +++++++++++++++++++
proxmox-auto-installer/src/lib.rs | 1 +
6 files changed, 190 insertions(+), 1 deletion(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils.rs
diff --git a/Makefile b/Makefile
index 3ac5769..e44450e 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,7 @@ PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
proxmox-tui-installer\
+ proxmox-fetch-answer\
proxmox-auto-installer
COMPILED_BINS := \
@@ -121,7 +122,8 @@ $(COMPILED_BINS): cargo-build
.PHONY: cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
- --package proxmox-auto-installer --bin proxmox-auto-installer $(CARGO_BUILD_ARGS)
+ --package proxmox-auto-installer --bin proxmox-auto-installer \
+ --bin proxmox-fetch-answer $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
new file mode 100644
index 0000000..a3681a2
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
@@ -0,0 +1,71 @@
+use anyhow::{anyhow, Error, Result};
+use log::{error, info, LevelFilter};
+use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger};
+use std::io::Write;
+use std::process::{Command, ExitCode, Stdio};
+
+static LOGGER: AutoInstLogger = AutoInstLogger;
+
+pub fn init_log() -> Result<()> {
+ AutoInstLogger::init("/tmp/fetch_answer.log")?;
+ log::set_logger(&LOGGER)
+ .map(|()| log::set_max_level(LevelFilter::Info))
+ .map_err(|err| anyhow!(err))
+}
+
+fn fetch_answer() -> Result<String> {
+ match FetchFromPartition::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file from partition failed: {err}"),
+ }
+ // TODO: add more options to get an answer file, e.g. download from url where url could be
+ // fetched via txt records on predefined subdomain, kernel param, dhcp option, ...
+
+ Err(Error::msg("Could not find any answer file!"))
+}
+
+fn main() -> ExitCode {
+ if let Err(err) = init_log() {
+ panic!("could not initialize logging: {err}");
+ }
+
+ info!("Fetching answer file");
+ let answer = match fetch_answer() {
+ Ok(answer) => answer,
+ Err(err) => {
+ error!("Aborting: {}", err);
+ return ExitCode::FAILURE;
+ }
+ };
+
+ let mut child = match Command::new("proxmox-auto-installer")
+ .stdout(Stdio::inherit())
+ .stdin(Stdio::piped())
+ .stderr(Stdio::null())
+ .spawn()
+ {
+ Ok(child) => child,
+ Err(err) => panic!("Failed to start automatic installation: {err}"),
+ };
+
+ let mut stdin = child.stdin.take().expect("Failed to open stdin");
+ std::thread::spawn(move || {
+ stdin
+ .write_all(answer.as_bytes())
+ .expect("Failed to write to stdin");
+ });
+
+ match child.wait() {
+ Ok(status) => {
+ if status.success() {
+ ExitCode::SUCCESS
+ } else {
+ ExitCode::FAILURE // Will be trapped
+ }
+ }
+ Err(err) => {
+ error!("Auto installer exited: {err}");
+ ExitCode::FAILURE
+ }
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
new file mode 100644
index 0000000..11d6937
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -0,0 +1,2 @@
+pub mod partition;
+mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/partition.rs b/proxmox-auto-installer/src/fetch_plugins/partition.rs
new file mode 100644
index 0000000..0c47a62
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/partition.rs
@@ -0,0 +1,32 @@
+use anyhow::{Error, Result};
+use std::{fs::read_to_string, path::Path};
+use log::info;
+
+use crate::fetch_plugins::utils::mount_proxmoxinst_part;
+
+static ANSWER_FILE: &str = "answer.toml";
+
+pub struct FetchFromPartition;
+
+impl FetchFromPartition {
+ /// Returns the contents of the answer file
+ pub fn get_answer() -> Result<String> {
+ info!("Checking for answer file on partition.");
+ let mount_path = mount_proxmoxinst_part()?;
+ let answer = Self::get_answer_file(&mount_path)?;
+ info!("Found answer file on partition.");
+ Ok(answer)
+ }
+
+ /// Searches for answer file and returns contents if found
+ fn get_answer_file(mount_path: &str) -> Result<String> {
+ let answer_path = Path::new(mount_path).join(ANSWER_FILE);
+ match answer_path.try_exists() {
+ Ok(true) => Ok(read_to_string(answer_path)?),
+ _ => Err(Error::msg(format!(
+ "could not find answer file expected at: {}",
+ answer_path.display()
+ ))),
+ }
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils.rs
new file mode 100644
index 0000000..f2a8e74
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils.rs
@@ -0,0 +1,81 @@
+use anyhow::{bail, Error, Result};
+use log::{info, warn};
+use std::{
+ fs::{self, create_dir_all},
+ path::{Path, PathBuf},
+ process::Command,
+};
+
+static ANSWER_MP: &str = "/mnt/answer";
+static PARTLABEL: &str = "proxmoxinst";
+static SEARCH_PATH: &str = "/dev/disk/by-label";
+
+/// Searches for upper and lower case existence of the partlabel in the search_path
+///
+/// # Arguemnts
+/// * `partlabel_source` - Partition Label, used as upper and lower case
+/// * `search_path` - Path where to search for the partiiton label
+pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<PathBuf> {
+ let partlabel = partlabel_source.to_uppercase();
+ let path = Path::new(search_path).join(&partlabel);
+ match path.try_exists() {
+ Ok(true) => {
+ info!("Found partition with label '{partlabel}'");
+ return Ok(path);
+ }
+ Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
+ }
+
+ let partlabel = partlabel_source.to_lowercase();
+ let path = Path::new(search_path).join(&partlabel);
+ match path.try_exists() {
+ Ok(true) => {
+ info!("Found partition with label '{partlabel}'");
+ return Ok(path);
+ }
+ Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
+ }
+ bail!("Could not detect upper or lower case labels for '{partlabel_source}'");
+}
+
+/// Will search and mount a partition/FS labeled proxmoxinst in lower or uppercase to ANSWER_MP;
+pub fn mount_proxmoxinst_part() -> Result<String> {
+ if let Ok(true) = check_if_mounted(ANSWER_MP) {
+ info!("Skipping: '{ANSWER_MP}' is already mounted.");
+ return Ok(ANSWER_MP.into());
+ }
+ let part_path = scan_partlabels(PARTLABEL, SEARCH_PATH)?;
+ info!("Mounting partition at {ANSWER_MP}");
+ // create dir for mountpoint
+ create_dir_all(ANSWER_MP)?;
+ match Command::new("mount")
+ .args(["-o", "ro"])
+ .arg(part_path)
+ .arg(ANSWER_MP)
+ .output()
+ {
+ Ok(output) => {
+ if output.status.success() {
+ Ok(ANSWER_MP.into())
+ } else {
+ warn!("Error mounting: {}", String::from_utf8(output.stderr)?);
+ Ok(ANSWER_MP.into())
+ }
+ }
+ Err(err) => Err(Error::msg(format!("Error mounting: {err}"))),
+ }
+}
+
+fn check_if_mounted(target_path: &str) -> Result<bool> {
+ let mounts = fs::read_to_string("/proc/mounts")?;
+ for line in mounts.lines() {
+ if let Some(mp) = line.split(' ').nth(1) {
+ if mp == target_path {
+ return Ok(true);
+ }
+ }
+ }
+ Ok(false)
+}
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 6636cc7..0a153b2 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,4 +1,5 @@
pub mod answer;
+pub mod fetch_plugins;
pub mod log;
pub mod udevinfo;
pub mod utils;
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 16/36] unconfigured: add proxauto as option to start auto installer
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (14 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 15/36] auto-installer: add fetch answer binary Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 17/36] auto-installer: use glob crate for pattern matching Aaron Lauterer
` (19 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
unconfigured.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/unconfigured.sh b/unconfigured.sh
index 2b371f0..f02336a 100755
--- a/unconfigured.sh
+++ b/unconfigured.sh
@@ -5,6 +5,7 @@ trap "err_reboot" ERR
# NOTE: we nowadays get exec'd by the initrd's PID 1, so we're the new PID 1
parse_cmdline() {
+ proxauto=0
proxdebug=0
proxtui=0
serial=0
@@ -17,6 +18,9 @@ parse_cmdline() {
proxtui)
proxtui=1
;;
+ proxauto)
+ proxauto=1
+ ;;
console=ttyS*)
serial=1
;;
@@ -224,6 +228,10 @@ setsid /sbin/agetty -a root --noclear tty3 &
if [ $proxtui -ne 0 ]; then
echo "Starting the TUI installer"
/usr/bin/proxmox-tui-installer 2>/dev/tty2
+elif [ $proxauto -ne 0 ]; then
+ /usr/bin/proxmox-low-level-installer dump-udev
+ echo "Starting automatic installation"
+ /usr/bin/proxmox-fetch-answer
else
echo "Starting the installer GUI - see tty2 (CTRL+ALT+F2) for any errors..."
xinit -- -dpi "$DPI" -s 0 >/dev/tty2 2>&1
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 17/36] auto-installer: use glob crate for pattern matching
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (15 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 16/36] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 18/36] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
` (18 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 1 +
proxmox-auto-installer/src/utils.rs | 46 +++++++++++------------------
2 files changed, 18 insertions(+), 29 deletions(-)
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index b1d3e7a..741794a 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -9,6 +9,7 @@ homepage = "https://www.proxmox.com"
[dependencies]
anyhow = "1.0"
+glob = "0.3"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index ae28b1e..6101d66 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -1,4 +1,5 @@
use anyhow::{bail, Result};
+use glob::Pattern;
use log::info;
use std::{
collections::BTreeMap,
@@ -15,29 +16,11 @@ use proxmox_installer_common::{
};
use serde::Deserialize;
-/// Supports the globbing character '*' at the beginning, end or both of the pattern.
-/// Globbing within the pattern is not supported
-fn find_with_glob(pattern: &str, value: &str) -> bool {
- let globbing_symbol = '*';
- let mut start_glob = false;
- let mut end_glob = false;
- let mut pattern = pattern;
-
- if pattern.starts_with(globbing_symbol) {
- start_glob = true;
- pattern = &pattern[1..];
- }
-
- if pattern.ends_with(globbing_symbol) {
- end_glob = true;
- pattern = &pattern[..pattern.len() - 1]
- }
-
- match (start_glob, end_glob) {
- (true, true) => value.contains(pattern),
- (true, false) => value.ends_with(pattern),
- (false, true) => value.starts_with(pattern),
- _ => value == pattern,
+fn find_with_glob(pattern: &str, value: &str) -> Result<bool> {
+ let p = Pattern::new(pattern)?;
+ match p.matches(value) {
+ true => Ok(true),
+ false => Ok(false),
}
}
@@ -75,7 +58,7 @@ fn get_single_udev_index(
'outer: for (dev, dev_values) in udev_list {
for (filter_key, filter_value) in &filter {
for (udev_key, udev_value) in dev_values {
- if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value)? {
dev_index = Some(dev.clone());
break 'outer; // take first match
}
@@ -100,7 +83,7 @@ fn get_matched_udev_indexes(
let mut did_match_all = true;
for (filter_key, filter_value) in &filter {
for (udev_key, udev_value) in dev_values {
- if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value)? {
did_match_once = true;
} else if udev_key == filter_key {
did_match_all = false;
@@ -416,9 +399,14 @@ mod tests {
#[test]
fn test_glob_patterns() {
let test_value = "foobar";
- assert_eq!(find_with_glob("*bar", test_value), true);
- assert_eq!(find_with_glob("foo*", test_value), true);
- assert_eq!(find_with_glob("foobar", test_value), true);
- assert_eq!(find_with_glob("oobar", test_value), false);
+ assert_eq!(find_with_glob("*bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("foo*", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("foobar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("oobar", test_value).unwrap(), false);
+ assert_eq!(find_with_glob("f*bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f?bar", test_value).unwrap(), false);
+ assert_eq!(find_with_glob("fo?bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f[!a]obar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f[oa]obar", test_value).unwrap(), true);
}
}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 18/36] auto-installer: utils: make get_udev_index functions public
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (16 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 17/36] auto-installer: use glob crate for pattern matching Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 19/36] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
` (17 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
because we will need to access them directly in the future from a
separate binary
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/utils.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index 6101d66..ea645ad 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -47,7 +47,7 @@ pub fn get_network_settings(
Ok(network_options)
}
-fn get_single_udev_index(
+pub fn get_single_udev_index(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
) -> Result<String> {
@@ -72,7 +72,7 @@ fn get_single_udev_index(
Ok(dev_index.unwrap())
}
-fn get_matched_udev_indexes(
+pub fn get_matched_udev_indexes(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
match_all: bool,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 19/36] auto-installer: add proxmox-autoinst-helper tool
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (17 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 18/36] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 20/36] common: add Display trait to ProxmoxProduct Aaron Lauterer
` (16 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
It can parse an answer file to check against syntax errors, test match
filters against the current hardware and list properties of the current
hardware to match against.
Since this tool should be able to run outside of the installer
environment, it does not rely on the device information provided by the
low-level installer. It instead fetches the list of disks and NICs by
itself.
The rules when a device is ignored, should match how the low-level
installer handles it.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 3 +-
proxmox-auto-installer/Cargo.toml | 2 +
proxmox-auto-installer/src/answer.rs | 3 +-
.../src/bin/proxmox-autoinst-helper.rs | 340 ++++++++++++++++++
4 files changed, 346 insertions(+), 2 deletions(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
diff --git a/Makefile b/Makefile
index e44450e..197a351 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,7 @@ PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
proxmox-tui-installer\
+ proxmox-autoinst-helper\
proxmox-fetch-answer\
proxmox-auto-installer
@@ -123,7 +124,7 @@ $(COMPILED_BINS): cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
- --bin proxmox-fetch-answer $(CARGO_BUILD_ARGS)
+ --bin proxmox-fetch-answer --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 741794a..bb0b49c 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -9,6 +9,7 @@ homepage = "https://www.proxmox.com"
[dependencies]
anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
glob = "0.3"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
@@ -16,3 +17,4 @@ serde_json = "1.0"
toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
+regex = "1.7"
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
index 0add95e..94cebb3 100644
--- a/proxmox-auto-installer/src/answer.rs
+++ b/proxmox-auto-installer/src/answer.rs
@@ -1,3 +1,4 @@
+use clap::ValueEnum;
use proxmox_installer_common::{
options::{BtrfsRaidLevel, FsType, ZfsChecksumOption, ZfsCompressOption, ZfsRaidLevel},
utils::{CidrAddress, Fqdn},
@@ -205,7 +206,7 @@ pub enum DiskSelection {
Selection(Vec<String>),
Filter(BTreeMap<String, String>),
}
-#[derive(Clone, Deserialize, Debug, PartialEq)]
+#[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)]
#[serde(rename_all = "lowercase")]
pub enum FilterMatch {
Any,
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
new file mode 100644
index 0000000..058d5ff
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
@@ -0,0 +1,340 @@
+use anyhow::{bail, Result};
+use clap::{Args, Parser, Subcommand, ValueEnum};
+use glob::Pattern;
+use regex::Regex;
+use serde::{Deserialize, Serialize};
+use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
+
+use proxmox_auto_installer::{
+ answer::Answer,
+ answer::FilterMatch,
+ utils::{get_matched_udev_indexes, get_single_udev_index},
+};
+
+/// This tool validates the format of an answer file. Additionally it can test match filters and
+/// print information on the properties to match against for the current hardware.
+#[derive(Parser, Debug)]
+#[command(author, version, about, long_about = None)]
+struct Cli {
+ #[command(subcommand)]
+ command: Commands,
+}
+
+#[derive(Subcommand, Debug)]
+enum Commands {
+ ValidateAnswer(CommandValidateAnswer),
+ Match(CommandMatch),
+ Info(CommandInfo),
+}
+
+/// Show device information that can be used for filters
+#[derive(Args, Debug)]
+struct CommandInfo {
+ /// For which device type information should be shown
+ #[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)]
+ device: AllDeviceTypes,
+}
+
+/// Test which devices the given filter matches against
+///
+/// Filters support the following syntax:
+/// ? Match a single character
+/// * Match any number of characters
+/// [a], [0-9] Specifc character or range of characters
+/// [!a] Negate a specific character of range
+///
+/// To avoid globbing characters being interpreted by the shell, use single quotes.
+/// Multiple filters can be defined.
+///
+/// Examples:
+/// Match disks against the serial number and device name, both must match:
+///
+/// proxmox-autoinst-helper match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*'
+#[derive(Args, Debug)]
+#[command(verbatim_doc_comment)]
+struct CommandMatch {
+ /// Device type to match the filter against
+ r#type: Devicetype,
+
+ /// Filter in the format KEY=VALUE where the key is the UDEV key and VALUE the filter string.
+ /// Multiple filters are possible, separated by a space.
+ filter: Vec<String>,
+
+ /// Defines if any filter or all filters must match.
+ #[arg(long, value_enum, default_value_t=FilterMatch::Any)]
+ filter_match: FilterMatch,
+}
+
+/// Validate if an answer file is formatted correctly.
+#[derive(Args, Debug)]
+struct CommandValidateAnswer {
+ /// Path to the answer file
+ path: PathBuf,
+ #[arg(short, long, default_value_t = false)]
+ debug: bool,
+}
+
+#[derive(Args, Debug)]
+struct GlobalOpts {
+ /// Output format
+ #[arg(long, short, value_enum)]
+ format: OutputFormat,
+}
+
+#[derive(Clone, Debug, ValueEnum, PartialEq)]
+enum AllDeviceTypes {
+ All,
+ Network,
+ Disk,
+}
+
+#[derive(Clone, Debug, ValueEnum)]
+enum Devicetype {
+ Network,
+ Disk,
+}
+
+#[derive(Clone, Debug, ValueEnum)]
+enum OutputFormat {
+ Pretty,
+ Json,
+}
+
+#[derive(Serialize)]
+struct Devs {
+ disks: Option<BTreeMap<String, BTreeMap<String, String>>>,
+ nics: Option<BTreeMap<String, BTreeMap<String, String>>>,
+}
+
+fn main() {
+ let args = Cli::parse();
+ let res = match &args.command {
+ Commands::Info(args) => info(args),
+ Commands::Match(args) => match_filter(args),
+ Commands::ValidateAnswer(args) => validate_answer(args),
+ };
+ if let Err(err) = res {
+ eprintln!("{err}");
+ std::process::exit(1);
+ }
+}
+
+fn info(args: &CommandInfo) -> Result<()> {
+ let mut devs = Devs {
+ disks: None,
+ nics: None,
+ };
+
+ if args.device == AllDeviceTypes::Network || args.device == AllDeviceTypes::All {
+ match get_nics() {
+ Ok(res) => devs.nics = Some(res),
+ Err(err) => bail!("Error getting NIC data: {err}"),
+ }
+ }
+ if args.device == AllDeviceTypes::Disk || args.device == AllDeviceTypes::All {
+ match get_disks() {
+ Ok(res) => devs.disks = Some(res),
+ Err(err) => bail!("Error getting disk data: {err}"),
+ }
+ }
+ println!("{}", serde_json::to_string_pretty(&devs).unwrap());
+ Ok(())
+}
+
+fn match_filter(args: &CommandMatch) -> Result<()> {
+ let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type {
+ Devicetype::Disk => get_disks().unwrap(),
+ Devicetype::Network => get_nics().unwrap(),
+ };
+ // parse filters
+
+ let mut filters: BTreeMap<String, String> = BTreeMap::new();
+
+ for f in &args.filter {
+ match f.split_once('=') {
+ Some((key, value)) => {
+ if key.is_empty() || value.is_empty() {
+ bail!("Filter key or value is empty in filter: '{f}'");
+ }
+ filters.insert(String::from(key), String::from(value));
+ }
+ None => {
+ bail!("Could not find separator '=' in filter: '{f}'");
+ }
+ }
+ }
+
+ // align return values
+ let result = match args.r#type {
+ Devicetype::Disk => {
+ get_matched_udev_indexes(filters, &devs, args.filter_match == FilterMatch::All)
+ }
+ Devicetype::Network => get_single_udev_index(filters, &devs).map(|r| vec![r]),
+ };
+
+ match result {
+ Ok(result) => println!("{}", serde_json::to_string_pretty(&result).unwrap()),
+ Err(err) => bail!("Error matching filters: {err}"),
+ }
+ Ok(())
+}
+
+fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
+ let mut file = match fs::File::open(&args.path) {
+ Ok(file) => file,
+ Err(err) => bail!(
+ "Opening answer file '{}' failed: {err}",
+ args.path.display()
+ ),
+ };
+ let mut contents = String::new();
+ if let Err(err) = file.read_to_string(&mut contents) {
+ bail!("Reading from file '{}' failed: {err}", args.path.display());
+ }
+
+ let answer: Answer = match toml::from_str(&contents) {
+ Ok(answer) => {
+ println!("The file was parsed successfully, no syntax errors found!");
+ answer
+ }
+ Err(err) => bail!("Error parsing answer file: {err}"),
+ };
+ if args.debug {
+ println!("Parsed data from answer file:\n{:#?}", answer);
+ }
+ Ok(())
+}
+
+fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
+ let unwantend_block_devs = vec![
+ "ram[0-9]*",
+ "loop[0-9]*",
+ "md[0-9]*",
+ "dm-*",
+ "fd[0-9]*",
+ "sr[0-9]*",
+ ];
+
+ // compile Regex here once and not inside the loop
+ let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?;
+ let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?;
+ let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?;
+
+ let re_name = Regex::new(r"(?m)^N: (.*)$")?;
+ let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
+
+ let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
+
+ 'outer: for entry in fs::read_dir("/sys/block")? {
+ let entry = entry.unwrap();
+ let filename = entry.file_name().into_string().unwrap();
+
+ for p in &unwantend_block_devs {
+ if Pattern::new(p)?.matches(&filename) {
+ continue 'outer;
+ }
+ }
+
+ let output = match get_udev_properties(&entry.path()) {
+ Ok(output) => output,
+ Err(err) => {
+ eprint!("{err}");
+ continue 'outer;
+ }
+ };
+
+ if !re_disk.is_match(&output) {
+ continue 'outer;
+ };
+ if re_cdrom.is_match(&output) {
+ continue 'outer;
+ };
+ if re_iso9660.is_match(&output) {
+ continue 'outer;
+ };
+
+ let mut name = filename;
+ if let Some(cap) = re_name.captures(&output) {
+ if let Some(res) = cap.get(1) {
+ name = String::from(res.as_str());
+ }
+ }
+
+ let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+
+ for line in output.lines() {
+ if let Some(caps) = re_props.captures(line) {
+ let key = String::from(caps.get(1).unwrap().as_str());
+ let value = String::from(caps.get(2).unwrap().as_str());
+ udev_props.insert(key, value);
+ }
+ }
+
+ disks.insert(name, udev_props);
+ }
+ Ok(disks)
+}
+
+#[derive(Deserialize, Debug)]
+struct IpLinksInfo {
+ ifname: String,
+}
+
+fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
+ let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
+ let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
+
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ for link in links {
+ let path = format!("/sys/class/net/{link}");
+
+ let output = match get_udev_properties(&PathBuf::from(path)) {
+ Ok(output) => output,
+ Err(err) => {
+ eprint!("{err}");
+ continue;
+ }
+ };
+
+ let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+
+ for line in output.lines() {
+ if let Some(caps) = re_props.captures(line) {
+ let key = String::from(caps.get(1).unwrap().as_str());
+ let value = String::from(caps.get(2).unwrap().as_str());
+ udev_props.insert(key, value);
+ }
+ }
+
+ nics.insert(link, udev_props);
+ }
+ Ok(nics)
+}
+
+fn get_udev_properties(path: &PathBuf) -> Result<String> {
+ let udev_output = Command::new("udevadm")
+ .arg("info")
+ .arg("--path")
+ .arg(path)
+ .arg("--query")
+ .arg("all")
+ .output()?;
+ if !udev_output.status.success() {
+ bail!("could not run udevadm successfully for {}", path.display());
+ }
+ Ok(String::from_utf8(udev_output.stdout)?)
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 20/36] common: add Display trait to ProxmoxProduct
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (18 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 19/36] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 21/36] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
` (15 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 8432a2c..25d0e9e 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -38,6 +38,16 @@ impl ProxmoxProduct {
}
}
+impl fmt::Display for ProxmoxProduct {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ match self {
+ Self::PVE => write!(f, "pve"),
+ Self::PMG => write!(f, "pmg"),
+ Self::PBS => write!(f, "pbs"),
+ }
+ }
+}
+
#[derive(Clone, Deserialize)]
pub struct ProductConfig {
pub fullname: String,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 21/36] auto-installer: fetch: add gathering of system identifiers and restructure code
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (19 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 20/36] common: add Display trait to ProxmoxProduct Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 22/36] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
` (14 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
They will be used as payload when POSTing a request for an answer file. The
idea is, that with this information, it should be possible to identify
the system and generate a matching answer file on the fly.
Many of these properties can also be found on the machine or packaging
of the machine and could therefore be scanned into a database.
Identifiers are the following properties from `dmidecode` sections 1, 2,
and 3:
* Asset Tag
* Product Name
* Serial Number
* SKU Number
* UUID
As well as a list of the MAC addresses of all the NICs and the product
type: pve, pmg, pbs.
Since we now have more than a simple utils.rs module in the fetch
plugins, it, and the additional fetch plugin utilities are placed in
their own directory.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/fetch_plugins/mod.rs | 2 +-
.../fetch_plugins/{utils.rs => utils/mod.rs} | 43 ++++++++--
.../src/fetch_plugins/utils/sysinfo.rs | 81 +++++++++++++++++++
3 files changed, 119 insertions(+), 7 deletions(-)
rename proxmox-auto-installer/src/fetch_plugins/{utils.rs => utils/mod.rs} (72%)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
index 11d6937..6f1e8a2 100644
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -1,2 +1,2 @@
pub mod partition;
-mod utils;
+pub mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
similarity index 72%
rename from proxmox-auto-installer/src/fetch_plugins/utils.rs
rename to proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
index f2a8e74..b3e9dad 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
@@ -1,5 +1,7 @@
-use anyhow::{bail, Error, Result};
+use anyhow::{Error, Result};
use log::{info, warn};
+use serde::Deserialize;
+use serde_json;
use std::{
fs::{self, create_dir_all},
path::{Path, PathBuf},
@@ -10,6 +12,8 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
+pub mod sysinfo;
+
/// Searches for upper and lower case existence of the partlabel in the search_path
///
/// # Arguemnts
@@ -20,10 +24,10 @@ pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<Path
let path = Path::new(search_path).join(&partlabel);
match path.try_exists() {
Ok(true) => {
- info!("Found partition with label '{partlabel}'");
+ info!("Found partition with label '{}'", partlabel);
return Ok(path);
}
- Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Ok(false) => info!("Did not detect partition with label '{}'", partlabel),
Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
}
@@ -31,13 +35,15 @@ pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<Path
let path = Path::new(search_path).join(&partlabel);
match path.try_exists() {
Ok(true) => {
- info!("Found partition with label '{partlabel}'");
+ info!("Found partition with label '{}'", partlabel);
return Ok(path);
}
- Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Ok(false) => info!("Did not detect partition with label '{}'", partlabel),
Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
}
- bail!("Could not detect upper or lower case labels for '{partlabel_source}'");
+ Err(Error::msg(format!(
+ "Could not detect upper or lower case labels for '{partlabel_source}'"
+ )))
}
/// Will search and mount a partition/FS labeled proxmoxinst in lower or uppercase to ANSWER_MP;
@@ -79,3 +85,28 @@ fn check_if_mounted(target_path: &str) -> Result<bool> {
}
Ok(false)
}
+
+#[derive(Deserialize, Debug)]
+struct IpLinksUdevInfo {
+ ifname: String,
+}
+
+/// Returns vec of usable NICs
+pub fn get_nic_list() -> Result<Vec<String>> {
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksUdevInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ Ok(links)
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
new file mode 100644
index 0000000..8c57283
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
@@ -0,0 +1,81 @@
+use anyhow::{bail, Result};
+use proxmox_installer_common::setup::SetupInfo;
+use serde::Serialize;
+use std::{collections::HashMap, fs, io, path::Path};
+
+use super::get_nic_list;
+
+const DMI_PATH: &str = "/sys/devices/virtual/dmi/id";
+
+pub fn get_sysinfo(pretty: bool) -> Result<String> {
+ let system_files = vec![
+ "product_serial",
+ "product_sku",
+ "product_uuid",
+ "product_name",
+ ];
+ let baseboard_files = vec!["board_asset_tag", "board_serial", "board_name"];
+ let chassis_files = vec!["chassis_serial", "chassis_sku", "chassis_asset_tag"];
+
+ let system = get_dmi_infos(system_files)?;
+ let baseboard = get_dmi_infos(baseboard_files)?;
+ let chassis = get_dmi_infos(chassis_files)?;
+
+ let mut mac_addresses: Vec<String> = Vec::new();
+ let links = get_nic_list()?;
+ for link in links {
+ let address = fs::read_to_string(format!("/sys/class/net/{link}/address"))?;
+ let address = String::from(address.trim());
+ mac_addresses.push(address);
+ }
+
+ let iso_info = Path::new("/run/proxmox-installer/iso-info.json");
+ let mut product = String::from("Not available. Would be one of the following: pve, pmg, pbs");
+ if iso_info.exists() {
+ let file = fs::File::open("/run/proxmox-installer/iso-info.json")?;
+ let reader = io::BufReader::new(file);
+ let setup_info: SetupInfo = serde_json::from_reader(reader)?;
+ product = setup_info.config.product.to_string();
+ }
+
+ let sysinfo = SysInfo {
+ product,
+ system,
+ baseboard,
+ chassis,
+ mac_addresses,
+ };
+ if pretty {
+ return Ok(serde_json::to_string_pretty(&sysinfo)?);
+ }
+ Ok(serde_json::to_string(&sysinfo)?)
+}
+
+#[derive(Debug, Serialize)]
+struct SysInfo {
+ product: String,
+ system: HashMap<String, String>,
+ baseboard: HashMap<String, String>,
+ chassis: HashMap<String, String>,
+ mac_addresses: Vec<String>,
+}
+
+fn get_dmi_infos(files: Vec<&str>) -> Result<HashMap<String, String>> {
+ let mut res: HashMap<String, String> = HashMap::new();
+
+ for file in files {
+ let path = format!("{DMI_PATH}/{file}");
+ let content = match fs::read_to_string(&path) {
+ Err(ref err) if err.kind() == std::io::ErrorKind::NotFound => continue,
+ Err(ref err) if err.kind() == std::io::ErrorKind::PermissionDenied => {
+ bail!("Could not read data. Are you running as root or with sudo?")
+ }
+ Err(err) => bail!("Error: '{err}' on '{path}'"),
+ Ok(content) => content.trim().into(),
+ };
+ let key = file.splitn(2, '_').last().unwrap();
+ res.insert(key.into(), content);
+ }
+
+ Ok(res)
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 22/36] auto-installer: helper: add subcommand to view indentifiers
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (20 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 21/36] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 23/36] auto-installer: fetch: add http post utility module Aaron Lauterer
` (13 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
It will collect the information from the current system and show the
payload of identifiers that will be send.
To avoid confusion, the subcommands for the device info and filter
matching have been renamed.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/bin/proxmox-autoinst-helper.rs | 54 +++++++++----------
1 file changed, 26 insertions(+), 28 deletions(-)
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
index 058d5ff..f0ee8f4 100644
--- a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
+++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
@@ -2,12 +2,13 @@ use anyhow::{bail, Result};
use clap::{Args, Parser, Subcommand, ValueEnum};
use glob::Pattern;
use regex::Regex;
-use serde::{Deserialize, Serialize};
+use serde::Serialize;
use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
+ fetch_plugins::utils::{sysinfo, get_nic_list},
utils::{get_matched_udev_indexes, get_single_udev_index},
};
@@ -23,13 +24,14 @@ struct Cli {
#[derive(Subcommand, Debug)]
enum Commands {
ValidateAnswer(CommandValidateAnswer),
- Match(CommandMatch),
- Info(CommandInfo),
+ DeviceMatch(CommandDeviceMatch),
+ DeviceInfo(CommandDeviceInfo),
+ Identifiers(CommandIdentifiers),
}
/// Show device information that can be used for filters
#[derive(Args, Debug)]
-struct CommandInfo {
+struct CommandDeviceInfo {
/// For which device type information should be shown
#[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)]
device: AllDeviceTypes,
@@ -52,7 +54,7 @@ struct CommandInfo {
/// proxmox-autoinst-helper match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*'
#[derive(Args, Debug)]
#[command(verbatim_doc_comment)]
-struct CommandMatch {
+struct CommandDeviceMatch {
/// Device type to match the filter against
r#type: Devicetype,
@@ -74,6 +76,11 @@ struct CommandValidateAnswer {
debug: bool,
}
+/// Show identifiers for the current machine. This information is part of the POST request to fetch
+/// an answer file.
+#[derive(Args, Debug)]
+struct CommandIdentifiers {}
+
#[derive(Args, Debug)]
struct GlobalOpts {
/// Output format
@@ -109,9 +116,10 @@ struct Devs {
fn main() {
let args = Cli::parse();
let res = match &args.command {
- Commands::Info(args) => info(args),
- Commands::Match(args) => match_filter(args),
Commands::ValidateAnswer(args) => validate_answer(args),
+ Commands::DeviceInfo(args) => info(args),
+ Commands::DeviceMatch(args) => match_filter(args),
+ Commands::Identifiers(args) => show_identifiers(args),
};
if let Err(err) = res {
eprintln!("{err}");
@@ -119,7 +127,7 @@ fn main() {
}
}
-fn info(args: &CommandInfo) -> Result<()> {
+fn info(args: &CommandDeviceInfo) -> Result<()> {
let mut devs = Devs {
disks: None,
nics: None,
@@ -141,7 +149,7 @@ fn info(args: &CommandInfo) -> Result<()> {
Ok(())
}
-fn match_filter(args: &CommandMatch) -> Result<()> {
+fn match_filter(args: &CommandDeviceMatch) -> Result<()> {
let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type {
Devicetype::Disk => get_disks().unwrap(),
Devicetype::Network => get_nics().unwrap(),
@@ -205,6 +213,14 @@ fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
Ok(())
}
+fn show_identifiers(_args: &CommandIdentifiers) -> Result<()> {
+ match sysinfo::get_sysinfo(true) {
+ Ok(res) => println!("{res}"),
+ Err(err) => eprintln!("Error fetching system identifiers: {err}"),
+ }
+ Ok(())
+}
+
fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let unwantend_block_devs = vec![
"ram[0-9]*",
@@ -275,30 +291,12 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
Ok(disks)
}
-#[derive(Deserialize, Debug)]
-struct IpLinksInfo {
- ifname: String,
-}
fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
- let ip_output = Command::new("/usr/sbin/ip")
- .arg("-j")
- .arg("link")
- .output()?;
- let parsed_links: Vec<IpLinksInfo> =
- serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
- let mut links: Vec<String> = Vec::new();
-
- for link in parsed_links {
- if link.ifname == *"lo" {
- continue;
- }
- links.push(link.ifname);
- }
-
+ let links = get_nic_list()?;
for link in links {
let path = format!("/sys/class/net/{link}");
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 23/36] auto-installer: fetch: add http post utility module
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (21 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 22/36] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 24/36] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
` (12 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
It sends a http(s) POST request with the sysinfo as payload and expects
an answer file in return.
In order to handle non FQDN URLs (e.g. IP addresses) and self signed
certificates, it can optionally take an SHA256 fingerprint of the
certificate. This can of course also be used to pin a certificate
explicitly, even if it would be in the trust chain.
A custom cert verifier for ureq / rustl was necessary to get cert
fingerprint matching to work.
If no fingerprint is proviced, we switch rustls to native-certs and
native-tls.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 6 ++
.../src/fetch_plugins/utils/mod.rs | 1 +
.../src/fetch_plugins/utils/post.rs | 94 +++++++++++++++++++
3 files changed, 101 insertions(+)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/post.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index bb0b49c..ac2f3a6 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -18,3 +18,9 @@ toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
regex = "1.7"
+ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
+rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
+rustls-native-certs = "0.6"
+native-tls = "0.2"
+sha2 = "0.10"
+hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
index b3e9dad..6b4c7db 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
@@ -12,6 +12,7 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
+pub mod post;
pub mod sysinfo;
/// Searches for upper and lower case existence of the partlabel in the search_path
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
new file mode 100644
index 0000000..193e920
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
@@ -0,0 +1,94 @@
+use anyhow::Result;
+use rustls::ClientConfig;
+use sha2::{Digest, Sha256};
+use std::sync::Arc;
+use ureq::{Agent, AgentBuilder};
+
+/// Issues a POST request with the payload (JSON). Optionally a SHA256 fingerprint can be used to
+/// check the cert against it, instead of the regular cert validation.
+/// To gather the sha256 fingerprint you can use the following command:
+/// ```no_compile
+/// openssl s_client -connect <host>:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin
+/// ```
+///
+/// # Arguemnts
+/// * `url` - URL to call
+/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional.
+/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string.
+pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> {
+ let answer ;
+
+ if let Some(fingerprint) = fingerprint {
+ let tls_config = ClientConfig::builder()
+ .with_safe_defaults()
+ .with_custom_certificate_verifier(VerifyCertFingerprint::new(fingerprint)?)
+ .with_no_client_auth();
+
+ let agent: Agent = AgentBuilder::new().tls_config(Arc::new(tls_config)).build();
+
+ answer = agent
+ .post(&url)
+ .set("Content-type", "application/json; charset=utf-")
+ .send_string(&payload)?
+ .into_string()?;
+ } else {
+ let mut roots = rustls::RootCertStore::empty();
+ for cert in rustls_native_certs::load_native_certs()? {
+ roots.add(&rustls::Certificate(cert.0)).unwrap();
+ }
+
+ let tls_config = rustls::ClientConfig::builder()
+ .with_safe_defaults()
+ .with_root_certificates(roots)
+ .with_no_client_auth();
+
+ let agent = AgentBuilder::new()
+ .tls_connector(Arc::new(native_tls::TlsConnector::new()?))
+ .tls_config(Arc::new(tls_config))
+ .build();
+ answer = agent
+ .post(&url)
+ .set("Content-type", "application/json; charset=utf-")
+ .timeout(std::time::Duration::from_secs(60))
+ .send_string(&payload)?
+ .into_string()?;
+ }
+ Ok(answer)
+}
+
+struct VerifyCertFingerprint {
+ cert_fingerprint: Vec<u8>,
+}
+
+impl VerifyCertFingerprint {
+ fn new<S: AsRef<str>>(cert_fingerprint: S) -> Result<std::sync::Arc<Self>> {
+ let cert_fingerprint = cert_fingerprint.as_ref();
+ let sanitized = cert_fingerprint.replace(':', "");
+ let decoded = hex::decode(sanitized)?;
+ Ok(std::sync::Arc::new(Self {
+ cert_fingerprint: decoded,
+ }))
+ }
+}
+
+impl rustls::client::ServerCertVerifier for VerifyCertFingerprint {
+ fn verify_server_cert(
+ &self,
+ end_entity: &rustls::Certificate,
+ _intermediates: &[rustls::Certificate],
+ _server_name: &rustls::ServerName,
+ _scts: &mut dyn Iterator<Item = &[u8]>,
+ _ocsp_response: &[u8],
+ _now: std::time::SystemTime,
+ ) -> Result<rustls::client::ServerCertVerified, rustls::Error> {
+ let mut hasher = Sha256::new();
+ hasher.update(end_entity);
+ let result = hasher.finalize();
+
+ if result.as_slice() == self.cert_fingerprint {
+ Ok(rustls::client::ServerCertVerified::assertion())
+ } else {
+ Err(rustls::Error::General("Fingerprint did not match!".into()))
+ }
+ }
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 24/36] auto-installer: fetch: add http plugin to fetch answer
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (22 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 23/36] auto-installer: fetch: add http post utility module Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 25/36] control: update build depends for auto installer Aaron Lauterer
` (11 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
This plugin will send a HTTP POST request with identifying sysinfo to
fetch an answer file. The provided sysinfo can be used to identify the
system and generate a matching answer file on demand.
The URL to send the request to, can be defined in two ways. Via a custom
DHCP option or a TXT record on a predefined subdomain, relative to the
search domain received via DHCP.
Additionally it is possible to specify a SHA256 SSL fingerprint. This
can be useful if a self-signed certificate is used or the URL is using
an IP address instead of an FQDN. Even with a trusted cert, it can be
used to pin this specific certificate.
The certificate fingerprint can either be placed on the `proxmoxinst`
partition and needs to be called `cert_fingerprint.txt`, or it can be
provided in a second custom DHCP option or a TXT record.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/bin/proxmox-fetch-answer.rs | 11 +-
.../src/fetch_plugins/http.rs | 190 ++++++++++++++++++
.../src/fetch_plugins/mod.rs | 1 +
unconfigured.sh | 9 +
4 files changed, 208 insertions(+), 3 deletions(-)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/http.rs
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
index a3681a2..6d42df2 100644
--- a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
+++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
@@ -1,6 +1,9 @@
use anyhow::{anyhow, Error, Result};
use log::{error, info, LevelFilter};
-use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger};
+use proxmox_auto_installer::{
+ fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition},
+ log::AutoInstLogger,
+};
use std::io::Write;
use std::process::{Command, ExitCode, Stdio};
@@ -18,8 +21,10 @@ fn fetch_answer() -> Result<String> {
Ok(answer) => return Ok(answer),
Err(err) => info!("Fetching answer file from partition failed: {err}"),
}
- // TODO: add more options to get an answer file, e.g. download from url where url could be
- // fetched via txt records on predefined subdomain, kernel param, dhcp option, ...
+ match FetchFromHTTP::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file via HTTP failed: {err}"),
+ }
Err(Error::msg("Could not find any answer file!"))
}
diff --git a/proxmox-auto-installer/src/fetch_plugins/http.rs b/proxmox-auto-installer/src/fetch_plugins/http.rs
new file mode 100644
index 0000000..4ac9afb
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/http.rs
@@ -0,0 +1,190 @@
+use anyhow::{bail, Error, Result};
+use log::info;
+use std::{
+ fs::{self, read_to_string},
+ path::Path,
+ process::Command,
+};
+
+use crate::fetch_plugins::utils::{post, sysinfo};
+
+use super::utils;
+
+static CERT_FINGERPRINT_FILE: &str = "cert_fingerprint.txt";
+static ANSWER_SUBDOMAIN: &str = "proxmoxinst";
+static ANSWER_SUBDOMAIN_FP: &str = "proxmoxinst-fp";
+
+// It is possible to set custom DHPC options. Option numbers 224 to 254 [0].
+// To use them with dhclient, we need to configure it to request them and what they should be
+// called.
+//
+// e.g. /etc/dhcp/dhclient.conf:
+// ```
+// option proxmoxinst-url code 250 = text;
+// option proxmoxinst-fp code 251 = text;
+// also request proxmoxinst-url, proxmoxinst-fp;
+// ```
+//
+// The results will end up in the /var/lib/dhcp/dhclient.leases file from where we can fetch them
+//
+// [0] https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml
+static DHCP_URL_OPTION: &str = "proxmoxinst-url";
+static DHCP_FP_OPTION: &str = "proxmoxinst-fp";
+static DHCP_LEASE_FILE: &str = "/var/lib/dhcp/dhclient.leases";
+
+pub struct FetchFromHTTP;
+
+impl FetchFromHTTP {
+ /// Will try to fetch the answer.toml by sending a HTTP POST request. The URL can be configured
+ /// either via DHCP or DNS.
+ /// DHCP options are checked first. The SSL certificate need to be either trusted by the root
+ /// certs or a SHA256 fingerprint needs to be provided. The SHA256 SSL fingerprint can either
+ /// be placed in a `cert_fingerprint.txt` file in the `proxmoxinst` partition, as DHCP option,
+ /// or as DNS TXT record. If provided, the `cert_fingerprint.txt` file has preference.
+ pub fn get_answer() -> Result<String> {
+ info!("Checking for certificate fingerprint in file.");
+ let mut fingerprint: Option<String> = match Self::get_cert_fingerprint_from_file() {
+ Ok(fp) => Some(fp),
+ Err(err) => {
+ info!("{err}");
+ None
+ }
+ };
+
+ let answer_url: String;
+
+ (answer_url, fingerprint) = match Self::fetch_dhcp(fingerprint.clone()) {
+ Ok((url, fp)) => (url, fp),
+ Err(err) => {
+ info!("{err}");
+ Self::fetch_dns(fingerprint.clone())?
+ }
+ };
+
+ if fingerprint.is_some() {
+ let fp = fingerprint.clone();
+ fs::write("/tmp/cert_fingerprint", fp.unwrap()).ok();
+ }
+
+ info!("Gathering system information.");
+ let payload = sysinfo::get_sysinfo(false)?;
+ info!("Sending POST request to '{answer_url}'.");
+ let answer = post::call(answer_url, fingerprint.as_deref(), payload)?;
+ Ok(answer)
+ }
+
+ /// Reads certificate fingerprint from file
+ pub fn get_cert_fingerprint_from_file() -> Result<String> {
+ let mount_path = utils::mount_proxmoxinst_part()?;
+ let cert_path = Path::new(mount_path.as_str()).join(CERT_FINGERPRINT_FILE);
+ match cert_path.try_exists() {
+ Ok(true) => {
+ info!("Found certifacte fingerprint file.");
+ Ok(fs::read_to_string(cert_path)?.trim().into())
+ }
+ _ => Err(Error::msg(format!(
+ "could not find cert fingerprint file expected at: {}",
+ cert_path.display()
+ ))),
+ }
+ }
+
+ /// Fetches search domain from resolv.conf file
+ fn get_search_domain() -> Result<String> {
+ info!("Retrieving default search domain.");
+ for line in read_to_string("/etc/resolv.conf")?.lines() {
+ if let Some((key, value)) = line.split_once(' ') {
+ if key == "search" {
+ return Ok(value.trim().into());
+ }
+ }
+ }
+ Err(Error::msg("Could not find search domain in resolv.conf."))
+ }
+
+ /// Runs a TXT DNS query on the domain provided
+ fn query_txt_record(query: String) -> Result<String> {
+ info!("Querying TXT record for '{query}'");
+ let url: String;
+ match Command::new("dig")
+ .args(["txt", "+short"])
+ .arg(&query)
+ .output()
+ {
+ Ok(output) => {
+ if output.status.success() {
+ url = String::from_utf8(output.stdout)?
+ .replace('"', "")
+ .trim()
+ .into();
+ if url.is_empty() {
+ bail!("Got empty response.");
+ }
+ } else {
+ bail!(
+ "Error querying DNS record '{query}' : {}",
+ String::from_utf8(output.stderr)?
+ );
+ }
+ }
+ Err(err) => bail!("Error querying DNS record '{query}': {err}"),
+ }
+ info!("Found: '{url}'");
+ Ok(url)
+ }
+
+ /// Tries to fetch answer URL and SSL fingerprint info from DNS
+ fn fetch_dns(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> {
+ let search_domain = Self::get_search_domain()?;
+
+ let answer_url = match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN}.{search_domain}"))
+ {
+ Ok(url) => url,
+ Err(err) => bail!("{err}"),
+ };
+
+ if fingerprint.is_none() {
+ fingerprint =
+ match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN_FP}.{search_domain}")) {
+ Ok(fp) => Some(fp),
+ Err(err) => {
+ info!("{err}");
+ None
+ }
+ };
+ }
+ Ok((answer_url, fingerprint))
+ }
+
+ /// Tries to fetch answer URL and SSL fingerprint info from DHCP options
+ fn fetch_dhcp(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> {
+ let leases = fs::read_to_string(DHCP_LEASE_FILE)?;
+
+ let mut answer_url: Option<String> = None;
+
+ let url_match = format!("option {DHCP_URL_OPTION}");
+ let fp_match = format!("option {DHCP_FP_OPTION}");
+
+ for line in leases.lines() {
+ if answer_url.is_none() && line.trim().starts_with(url_match.as_str()) {
+ answer_url = Self::strip_dhcp_option(line.split(' ').nth_back(0));
+ }
+ if fingerprint.is_none() && line.trim().starts_with(fp_match.as_str()) {
+ fingerprint = Self::strip_dhcp_option(line.split(' ').nth_back(0));
+ }
+ }
+
+ let answer_url = match answer_url {
+ None => bail!("No DHCP option found for fetch URL."),
+ Some(url) => url,
+ };
+
+ Ok((answer_url, fingerprint))
+ }
+
+ /// Clean DHCP option string
+ fn strip_dhcp_option(value: Option<&str>) -> Option<String> {
+ // value is expected to be in format: "value";
+ value.map(|value| String::from(&value[1..value.len() - 2]))
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
index 6f1e8a2..354fa7e 100644
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -1,2 +1,3 @@
+pub mod http;
pub mod partition;
pub mod utils;
diff --git a/unconfigured.sh b/unconfigured.sh
index f02336a..dbdb027 100755
--- a/unconfigured.sh
+++ b/unconfigured.sh
@@ -212,6 +212,15 @@ if [ $proxdebug -ne 0 ]; then
debugsh || true
fi
+# add custom DHCP options for auto installer
+if [ $proxauto -ne 0 ]; then
+ cat >> /etc/dhcp/dhclient.conf <<EOF
+option proxmoxinst-url code 250 = text;
+option proxmoxinst-fp code 251 = text;
+also request proxmoxinst-url, proxmoxinst-fp;
+EOF
+fi
+
# try to get ip config with dhcp
echo -n "Attempting to get DHCP leases... "
dhclient -v
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 25/36] control: update build depends for auto installer
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (23 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 24/36] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 26/36] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
` (10 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
debian/control | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/debian/control b/debian/control
index 3ca208b..1326400 100644
--- a/debian/control
+++ b/debian/control
@@ -8,10 +8,20 @@ Build-Depends: cargo:native,
libgtk3-perl,
libpve-common-perl,
librsvg2-bin,
+ librust-anyhow-dev,
+ librust-clap-dev,
librust-cursive+termion-backend-dev (>= 0.20.0),
+ librust-glob-dev,
+ librust-hex-dev,
+ librust-native-tls-dev,
librust-regex-1+default-dev (>= 1.7~~),
+ librust-rustls-dev,
+ librust-rustls-native-certs-dev,
librust-serde-1+default-dev,
librust-serde-json-1+default-dev,
+ librust-sha2-dev,
+ librust-ureq-dev,
+ librust-toml-dev,
libtest-mockmodule-perl,
perl,
rustc:native,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 26/36] auto installer: factor out fetch-answer and autoinst-helper
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (24 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 25/36] control: update build depends for auto installer Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 27/36] low-level: write low level config to /tmp Aaron Lauterer
` (9 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
Putting proxmox-fetch-answer into it's own crate, will keep the use of
OpenSSL localized to where we need it. Otherwise building other binaries
will always depend on OpenSSL as well, even without actually needing it.
Having a dedicated crate for the proxmox-autoinst-helper should make it
easier to build it independently to have it available outside of the
install environment.
The fetch plugins have been moved to the proxmox-fetch-answer crate,
except for the 'get_nic_list' function and 'sysinfo.rs'. Since both are
also needed by the proxmox-autoinst-helper, they are kept in the
proxmox-auto-installer crate.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 2 ++
Makefile | 5 +++-
proxmox-auto-installer/Cargo.toml | 6 ----
.../src/fetch_plugins/mod.rs | 3 --
proxmox-auto-installer/src/lib.rs | 2 +-
.../src/{fetch_plugins/utils => }/sysinfo.rs | 2 +-
proxmox-auto-installer/src/utils.rs | 25 ++++++++++++++++
proxmox-autoinst-helper/Cargo.toml | 21 +++++++++++++
.../src/main.rs | 5 ++--
proxmox-fetch-answer/Cargo.toml | 22 ++++++++++++++
.../src/fetch_plugins/http.rs | 3 +-
proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 ++
.../src/fetch_plugins/partition.rs | 2 +-
.../src/fetch_plugins/utils/mod.rs | 30 +------------------
.../src/fetch_plugins/utils/post.rs | 2 +-
.../src/main.rs | 8 ++---
16 files changed, 90 insertions(+), 51 deletions(-)
delete mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
rename proxmox-auto-installer/src/{fetch_plugins/utils => }/sysinfo.rs (98%)
create mode 100644 proxmox-autoinst-helper/Cargo.toml
rename proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs => proxmox-autoinst-helper/src/main.rs (98%)
create mode 100644 proxmox-fetch-answer/Cargo.toml
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/http.rs (98%)
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/partition.rs (100%)
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/utils/mod.rs (81%)
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/utils/post.rs (99%)
rename proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs => proxmox-fetch-answer/src/main.rs (93%)
diff --git a/Cargo.toml b/Cargo.toml
index 7017ac5..b694d5b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,8 @@
[workspace]
members = [
"proxmox-auto-installer",
+ "proxmox-autoinst-helper",
+ "proxmox-fetch-answer",
"proxmox-installer-common",
"proxmox-tui-installer",
]
diff --git a/Makefile b/Makefile
index 197a351..e32d28f 100644
--- a/Makefile
+++ b/Makefile
@@ -52,6 +52,8 @@ $(BUILDDIR):
proxinstall \
proxmox-low-level-installer \
proxmox-auto-installer/ \
+ proxmox-autoinst-helper/ \
+ proxmox-fetch-answer/ \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
@@ -124,7 +126,8 @@ $(COMPILED_BINS): cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
- --bin proxmox-fetch-answer --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
+ --package proxmox-fetch-answer --bin proxmox-fetch-answer \
+ --package proxmox-autoinst-helper --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index ac2f3a6..bb0b49c 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -18,9 +18,3 @@ toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
regex = "1.7"
-ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
-rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
-rustls-native-certs = "0.6"
-native-tls = "0.2"
-sha2 = "0.10"
-hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
deleted file mode 100644
index 354fa7e..0000000
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ /dev/null
@@ -1,3 +0,0 @@
-pub mod http;
-pub mod partition;
-pub mod utils;
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 0a153b2..3bdf0b5 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,5 +1,5 @@
pub mod answer;
-pub mod fetch_plugins;
pub mod log;
+pub mod sysinfo;
pub mod udevinfo;
pub mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/sysinfo.rs
similarity index 98%
rename from proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
rename to proxmox-auto-installer/src/sysinfo.rs
index 8c57283..f5de214 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
+++ b/proxmox-auto-installer/src/sysinfo.rs
@@ -3,7 +3,7 @@ use proxmox_installer_common::setup::SetupInfo;
use serde::Serialize;
use std::{collections::HashMap, fs, io, path::Path};
-use super::get_nic_list;
+use crate::utils::get_nic_list;
const DMI_PATH: &str = "/sys/devices/virtual/dmi/id";
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index ea645ad..ff90ae8 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -72,6 +72,31 @@ pub fn get_single_udev_index(
Ok(dev_index.unwrap())
}
+#[derive(Deserialize, Debug)]
+struct IpLinksUdevInfo {
+ ifname: String,
+}
+
+/// Returns vec of usable NICs
+pub fn get_nic_list() -> Result<Vec<String>> {
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksUdevInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ Ok(links)
+}
+
pub fn get_matched_udev_indexes(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
diff --git a/proxmox-autoinst-helper/Cargo.toml b/proxmox-autoinst-helper/Cargo.toml
new file mode 100644
index 0000000..2a88c0f
--- /dev/null
+++ b/proxmox-autoinst-helper/Cargo.toml
@@ -0,0 +1,21 @@
+[package]
+name = "proxmox-autoinst-helper"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+glob = "0.3"
+proxmox-auto-installer = { path = "../proxmox-auto-installer" }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+toml = "0.7"
+log = "0.4.20"
+regex = "1.7"
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-autoinst-helper/src/main.rs
similarity index 98%
rename from proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
rename to proxmox-autoinst-helper/src/main.rs
index f0ee8f4..fe1cbec 100644
--- a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
+++ b/proxmox-autoinst-helper/src/main.rs
@@ -8,8 +8,8 @@ use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
- fetch_plugins::utils::{sysinfo, get_nic_list},
- utils::{get_matched_udev_indexes, get_single_udev_index},
+ sysinfo,
+ utils::{get_matched_udev_indexes, get_nic_list, get_single_udev_index},
};
/// This tool validates the format of an answer file. Additionally it can test match filters and
@@ -291,7 +291,6 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
Ok(disks)
}
-
fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
diff --git a/proxmox-fetch-answer/Cargo.toml b/proxmox-fetch-answer/Cargo.toml
new file mode 100644
index 0000000..fbcca46
--- /dev/null
+++ b/proxmox-fetch-answer/Cargo.toml
@@ -0,0 +1,22 @@
+[package]
+name = "proxmox-fetch-answer"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+proxmox-auto-installer = { path = "../proxmox-auto-installer" }
+log = "0.4.20"
+ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
+rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
+rustls-native-certs = "0.6"
+native-tls = "0.2"
+sha2 = "0.10"
+hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/http.rs b/proxmox-fetch-answer/src/fetch_plugins/http.rs
similarity index 98%
rename from proxmox-auto-installer/src/fetch_plugins/http.rs
rename to proxmox-fetch-answer/src/fetch_plugins/http.rs
index 4ac9afb..5772c42 100644
--- a/proxmox-auto-installer/src/fetch_plugins/http.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/http.rs
@@ -6,7 +6,8 @@ use std::{
process::Command,
};
-use crate::fetch_plugins::utils::{post, sysinfo};
+use crate::fetch_plugins::utils::post;
+use proxmox_auto_installer::sysinfo;
use super::utils;
diff --git a/proxmox-fetch-answer/src/fetch_plugins/mod.rs b/proxmox-fetch-answer/src/fetch_plugins/mod.rs
new file mode 100644
index 0000000..d148e4c
--- /dev/null
+++ b/proxmox-fetch-answer/src/fetch_plugins/mod.rs
@@ -0,0 +1,3 @@
+pub(crate) mod http;
+pub(crate) mod partition;
+pub(crate) mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
similarity index 100%
rename from proxmox-auto-installer/src/fetch_plugins/partition.rs
rename to proxmox-fetch-answer/src/fetch_plugins/partition.rs
index 0c47a62..dbe5dda 100644
--- a/proxmox-auto-installer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -1,6 +1,6 @@
use anyhow::{Error, Result};
-use std::{fs::read_to_string, path::Path};
use log::info;
+use std::{fs::read_to_string, path::Path};
use crate::fetch_plugins::utils::mount_proxmoxinst_part;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
similarity index 81%
rename from proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
rename to proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
index 6b4c7db..e5ea4b8 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
@@ -1,7 +1,5 @@
use anyhow::{Error, Result};
use log::{info, warn};
-use serde::Deserialize;
-use serde_json;
use std::{
fs::{self, create_dir_all},
path::{Path, PathBuf},
@@ -12,8 +10,7 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
-pub mod post;
-pub mod sysinfo;
+pub(crate) mod post;
/// Searches for upper and lower case existence of the partlabel in the search_path
///
@@ -86,28 +83,3 @@ fn check_if_mounted(target_path: &str) -> Result<bool> {
}
Ok(false)
}
-
-#[derive(Deserialize, Debug)]
-struct IpLinksUdevInfo {
- ifname: String,
-}
-
-/// Returns vec of usable NICs
-pub fn get_nic_list() -> Result<Vec<String>> {
- let ip_output = Command::new("/usr/sbin/ip")
- .arg("-j")
- .arg("link")
- .output()?;
- let parsed_links: Vec<IpLinksUdevInfo> =
- serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
- let mut links: Vec<String> = Vec::new();
-
- for link in parsed_links {
- if link.ifname == *"lo" {
- continue;
- }
- links.push(link.ifname);
- }
-
- Ok(links)
-}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
similarity index 99%
rename from proxmox-auto-installer/src/fetch_plugins/utils/post.rs
rename to proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
index 193e920..89658f3 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
@@ -16,7 +16,7 @@ use ureq::{Agent, AgentBuilder};
/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional.
/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string.
pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> {
- let answer ;
+ let answer;
if let Some(fingerprint) = fingerprint {
let tls_config = ClientConfig::builder()
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-fetch-answer/src/main.rs
similarity index 93%
rename from proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
rename to proxmox-fetch-answer/src/main.rs
index 6d42df2..8c762e9 100644
--- a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
+++ b/proxmox-fetch-answer/src/main.rs
@@ -1,12 +1,12 @@
use anyhow::{anyhow, Error, Result};
+use fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition};
use log::{error, info, LevelFilter};
-use proxmox_auto_installer::{
- fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition},
- log::AutoInstLogger,
-};
+use proxmox_auto_installer::log::AutoInstLogger;
use std::io::Write;
use std::process::{Command, ExitCode, Stdio};
+mod fetch_plugins;
+
static LOGGER: AutoInstLogger = AutoInstLogger;
pub fn init_log() -> Result<()> {
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 27/36] low-level: write low level config to /tmp
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (25 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 26/36] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
@ 2024-04-17 12:30 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 28/36] common: add deserializer for FsType Aaron Lauterer
` (8 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:30 UTC (permalink / raw)
To: pve-devel
This helps to know how the system was set up in steps after the
installation. For example in debug mode or when using post commands in
the automatic/unattended installation.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-low-level-installer | 1 +
1 file changed, 1 insertion(+)
diff --git a/proxmox-low-level-installer b/proxmox-low-level-installer
index 54f689a..935bf17 100755
--- a/proxmox-low-level-installer
+++ b/proxmox-low-level-installer
@@ -69,6 +69,7 @@ sub read_and_merge_config {
Proxmox::Install::Config::merge($config);
log_info("got installation config: ". to_json(Proxmox::Install::Config::get(), { utf8 => 1, canonical => 1 }) ."\n");
+ file_write_all("/tmp/low-level-config.json", to_json(Proxmox::Install::Config::get()));
}
sub send_reboot_ui_message {
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 28/36] common: add deserializer for FsType
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (26 preceding siblings ...)
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 27/36] low-level: write low level config to /tmp Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 29/36] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
` (7 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/Cargo.toml | 1 +
proxmox-installer-common/src/options.rs | 10 ++++++---
proxmox-installer-common/src/setup.rs | 30 ++++++++++++++++++++++---
3 files changed, 35 insertions(+), 6 deletions(-)
diff --git a/proxmox-installer-common/Cargo.toml b/proxmox-installer-common/Cargo.toml
index bde5457..70f828a 100644
--- a/proxmox-installer-common/Cargo.toml
+++ b/proxmox-installer-common/Cargo.toml
@@ -8,5 +8,6 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+regex = "1.7"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
diff --git a/proxmox-installer-common/src/options.rs b/proxmox-installer-common/src/options.rs
index 1efac66..1e782f9 100644
--- a/proxmox-installer-common/src/options.rs
+++ b/proxmox-installer-common/src/options.rs
@@ -1,6 +1,6 @@
+use serde::Deserialize;
use std::net::{IpAddr, Ipv4Addr};
use std::{cmp, fmt};
-use serde::Deserialize;
use crate::setup::{
LocaleInfo, NetworkInfo, ProductConfig, ProxmoxProduct, RuntimeInfo, SetupInfo,
@@ -32,8 +32,11 @@ pub enum ZfsRaidLevel {
Raid0,
Raid1,
Raid10,
+ #[serde(rename = "raidz-1")]
RaidZ,
+ #[serde(rename = "raidz-2")]
RaidZ2,
+ #[serde(rename = "raidz-3")]
RaidZ3,
}
@@ -51,7 +54,8 @@ impl fmt::Display for ZfsRaidLevel {
}
}
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum FsType {
Ext4,
Xfs,
@@ -226,7 +230,7 @@ pub enum AdvancedBootdiskOptions {
Btrfs(BtrfsBootdiskOptions),
}
-#[derive(Clone, Debug, PartialEq)]
+#[derive(Clone, Debug, Deserialize, PartialEq)]
pub struct Disk {
pub index: String,
pub path: String,
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 25d0e9e..c580477 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -155,7 +155,7 @@ pub fn installer_setup(in_test_mode: bool) -> Result<(SetupInfo, LocaleInfo, Run
}
}
-#[derive(Serialize)]
+#[derive(Debug, Deserialize, Serialize)]
pub struct InstallZfsOption {
pub ashift: usize,
#[serde(serialize_with = "serialize_as_display")]
@@ -402,11 +402,11 @@ pub fn spawn_low_level_installer(test_mode: bool) -> io::Result<process::Child>
}
/// See Proxmox::Install::Config
-#[derive(Serialize)]
+#[derive(Debug, Deserialize, Serialize)]
pub struct InstallConfig {
pub autoreboot: usize,
- #[serde(serialize_with = "serialize_fstype")]
+ #[serde(serialize_with = "serialize_fstype", deserialize_with = "deserialize_fs_type")]
pub filesys: FsType,
pub hdsize: f64,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -481,3 +481,27 @@ where
serializer.collect_str(value)
}
+
+pub fn deserialize_fs_type<'de, D>(deserializer: D) -> Result<FsType, D::Error>
+where
+ D: Deserializer<'de>,
+{
+ use FsType::*;
+ let de_fs: String = Deserialize::deserialize(deserializer)?;
+
+ println!("deserializing fstype");
+ match de_fs.as_str() {
+ "ext4" => Ok(Ext4),
+ "xfs" => Ok(Xfs),
+ "zfs (RAID0)" => Ok(Zfs(ZfsRaidLevel::Raid0)),
+ "zfs (RAID1)" => Ok(Zfs(ZfsRaidLevel::Raid1)),
+ "zfs (RAID10)" => Ok(Zfs(ZfsRaidLevel::Raid10)),
+ "zfs (RAIDZ-1)" => Ok(Zfs(ZfsRaidLevel::RaidZ)),
+ "zfs (RAIDZ-2)" => Ok(Zfs(ZfsRaidLevel::RaidZ2)),
+ "zfs (RAIDZ-3)" => Ok(Zfs(ZfsRaidLevel::RaidZ3)),
+ "btrfs (RAID0)" => Ok(Btrfs(BtrfsRaidLevel::Raid0)),
+ "btrfs (RAID1)" => Ok(Btrfs(BtrfsRaidLevel::Raid1)),
+ "btrfs (RAID10)" => Ok(Btrfs(BtrfsRaidLevel::Raid10)),
+ _ => Err(de::Error::custom("could not find file system: {de_fs}"))
+ }
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 29/36] common: skip target_hd when deserializing InstallConfig
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (27 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 28/36] common: add deserializer for FsType Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 30/36] add proxmox-chroot utility Aaron Lauterer
` (6 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
as only the 'path' property is serialized -> deserialization is
problematic. The information would be present in the 'run-env-info-json',
but for now there is no need for it in any code that deserializes the
low-level config. Therefore we are currently skipping it on
deserialization
If we need it in the future, we need to think about how to handle the
deserialization.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index c580477..2454cbd 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -423,7 +423,12 @@ pub struct InstallConfig {
#[serde(
serialize_with = "serialize_disk_opt",
- skip_serializing_if = "Option::is_none"
+ skip_serializing_if = "Option::is_none",
+ // only the 'path' property is serialized -> deserialization is problematic
+ // The information would be present in the 'run-env-info-json', but for now there is no
+ // need for it in any code that deserializes the low-level config. Therefore we are
+ // currently skipping it on deserialization
+ skip_deserializing
)]
pub target_hd: Option<Disk>,
#[serde(skip_serializing_if = "BTreeMap::is_empty")]
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 30/36] add proxmox-chroot utility
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (28 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 29/36] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 31/36] auto-installer: answer: deny unknown fields Aaron Lauterer
` (5 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
it is meant as a helper utility to prepare an installation for chroot
and clean up afterwards
It tries to determine the used FS from the previous installation, will
do what is necessary to mount/import the root FS to /target. It then
will set up all bind mounts.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 1 +
Makefile | 5 +-
proxmox-chroot/Cargo.toml | 16 ++
proxmox-chroot/src/main.rs | 356 +++++++++++++++++++++++++++++++++++++
4 files changed, 377 insertions(+), 1 deletion(-)
create mode 100644 proxmox-chroot/Cargo.toml
create mode 100644 proxmox-chroot/src/main.rs
diff --git a/Cargo.toml b/Cargo.toml
index b694d5b..b3afc7c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -2,6 +2,7 @@
members = [
"proxmox-auto-installer",
"proxmox-autoinst-helper",
+ "proxmox-chroot",
"proxmox-fetch-answer",
"proxmox-installer-common",
"proxmox-tui-installer",
diff --git a/Makefile b/Makefile
index e32d28f..d69dc6f 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,7 @@ INSTALLER_SOURCES=$(shell git ls-files) country.dat
PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
+ proxmox-chroot\
proxmox-tui-installer\
proxmox-autoinst-helper\
proxmox-fetch-answer\
@@ -54,6 +55,7 @@ $(BUILDDIR):
proxmox-auto-installer/ \
proxmox-autoinst-helper/ \
proxmox-fetch-answer/ \
+ proxmox-chroot \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
@@ -127,7 +129,8 @@ cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
--package proxmox-fetch-answer --bin proxmox-fetch-answer \
- --package proxmox-autoinst-helper --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
+ --package proxmox-autoinst-helper --bin proxmox-autoinst-helper \
+ --package proxmox-chroot --bin proxmox-chroot $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-chroot/Cargo.toml b/proxmox-chroot/Cargo.toml
new file mode 100644
index 0000000..43b96ff
--- /dev/null
+++ b/proxmox-chroot/Cargo.toml
@@ -0,0 +1,16 @@
+[package]
+name = "proxmox-chroot"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+nix = "0.26.1"
+proxmox-installer-common = { path = "../proxmox-installer-common" }
+regex = "1.7"
+serde_json = "1.0"
diff --git a/proxmox-chroot/src/main.rs b/proxmox-chroot/src/main.rs
new file mode 100644
index 0000000..c1a4785
--- /dev/null
+++ b/proxmox-chroot/src/main.rs
@@ -0,0 +1,356 @@
+use std::{fs, io, path, process::Command};
+
+use anyhow::{bail, Result};
+use clap::{Args, Parser, Subcommand, ValueEnum};
+use nix::mount::{mount, umount, MsFlags};
+use proxmox_installer_common::{
+ options::FsType,
+ setup::{InstallConfig, SetupInfo},
+};
+use regex::Regex;
+
+const ANSWER_MP: &str = "answer";
+static BINDMOUNTS: [&str; 4] = ["dev", "proc", "run", "sys"];
+const TARGET_DIR: &str = "/target";
+const ZPOOL_NAME: &str = "rpool";
+
+/// Helper tool to prepare eveything to `chroot` into an installation
+#[derive(Parser, Debug)]
+#[command(author, version, about, long_about = None)]
+struct Cli {
+ #[command(subcommand)]
+ command: Commands,
+}
+
+#[derive(Subcommand, Debug)]
+enum Commands {
+ Prepare(CommandPrepare),
+ Cleanup(CommandCleanup),
+}
+
+/// Mount the root file system and bind mounts in preparation to chroot into the installation
+#[derive(Args, Debug)]
+struct CommandPrepare {
+ /// Filesystem used for the installation. Will try to automatically detect it after a
+ /// successful installation.
+ #[arg(short, long, value_enum)]
+ filesystem: Option<Filesystems>,
+
+ /// Numerical ID of `rpool` ZFS pool to import. Needed if multiple pools of name `rpool` are present.
+ #[arg(long)]
+ rpool_id: Option<u64>,
+
+ /// UUID of the BTRFS file system to mount. Needed if multiple BTRFS file systems are present.
+ #[arg(long)]
+ btrfs_uuid: Option<String>,
+}
+
+/// Unmount everything. Use once done with chroot.
+#[derive(Args, Debug)]
+struct CommandCleanup {
+ /// Filesystem used for the installation. Will try to automatically detect it by default.
+ #[arg(short, long, value_enum)]
+ filesystem: Option<Filesystems>,
+}
+
+#[derive(Copy, Clone, Debug, ValueEnum)]
+enum Filesystems {
+ Zfs,
+ Ext4,
+ Xfs,
+ Btrfs,
+}
+
+impl From<FsType> for Filesystems {
+ fn from(fs: FsType) -> Self {
+ match fs {
+ FsType::Xfs => Self::Xfs,
+ FsType::Ext4 => Self::Ext4,
+ FsType::Zfs(_) => Self::Zfs,
+ FsType::Btrfs(_) => Self::Btrfs,
+ }
+ }
+}
+
+fn main() {
+ let args = Cli::parse();
+ let res = match &args.command {
+ Commands::Prepare(args) => prepare(args),
+ Commands::Cleanup(args) => cleanup(args),
+ };
+ if let Err(err) = res {
+ eprintln!("{err}");
+ std::process::exit(1);
+ }
+}
+
+fn prepare(args: &CommandPrepare) -> Result<()> {
+ let fs = get_fs(args.filesystem)?;
+
+ fs::create_dir_all(TARGET_DIR)?;
+
+ match fs {
+ Filesystems::Zfs => mount_zpool(args.rpool_id)?,
+ Filesystems::Xfs => mount_fs()?,
+ Filesystems::Ext4 => mount_fs()?,
+ Filesystems::Btrfs => mount_btrfs(args.btrfs_uuid.clone())?,
+ }
+
+ if let Err(e) = bindmount() {
+ eprintln!("{e}")
+ }
+
+ println!("Done. You can now use 'chroot /target /bin/bash'!");
+ Ok(())
+}
+
+fn cleanup(args: &CommandCleanup) -> Result<()> {
+ let fs = get_fs(args.filesystem)?;
+
+ if let Err(e) = bind_umount() {
+ eprintln!("{e}")
+ }
+
+ match fs {
+ Filesystems::Zfs => umount_zpool(),
+ Filesystems::Xfs => umount_fs()?,
+ Filesystems::Ext4 => umount_fs()?,
+ _ => (),
+ }
+
+ println!("Chroot cleanup done. You can now reboot or leave the shell.");
+ Ok(())
+}
+
+fn get_fs(filesystem: Option<Filesystems>) -> Result<Filesystems> {
+ let fs = match filesystem {
+ None => {
+ let low_level_config = match get_low_level_config() {
+ Ok(c) => c,
+ Err(_) => bail!("Could not fetch config from previous installation. Please specify file system with -f."),
+ };
+ Filesystems::from(low_level_config.filesys)
+ }
+ Some(fs) => fs,
+ };
+
+ Ok(fs)
+}
+
+fn get_low_level_config() -> Result<InstallConfig> {
+ let file = fs::File::open("/tmp/low-level-config.json")?;
+ let reader = io::BufReader::new(file);
+ let config: InstallConfig = serde_json::from_reader(reader)?;
+ Ok(config)
+}
+
+fn get_iso_info() -> Result<SetupInfo> {
+ let file = fs::File::open("/run/proxmox-installer/iso-info.json")?;
+ let reader = io::BufReader::new(file);
+ let setup_info: SetupInfo = serde_json::from_reader(reader)?;
+ Ok(setup_info)
+}
+
+fn mount_zpool(pool_id: Option<u64>) -> Result<()> {
+ println!("importing ZFS pool to {TARGET_DIR}");
+ let mut import = Command::new("zpool");
+ import.arg("import").args(["-R", TARGET_DIR]);
+ match pool_id {
+ None => {
+ import.arg(ZPOOL_NAME);
+ }
+ Some(id) => {
+ import.arg(id.to_string());
+ }
+ }
+ match import.status() {
+ Ok(s) if !s.success() => bail!("Could not import ZFS pool. Abort!"),
+ _ => (),
+ }
+ println!("successfully imported ZFS pool to {TARGET_DIR}");
+ Ok(())
+}
+
+fn umount_zpool() {
+ match Command::new("zpool").arg("export").arg(ZPOOL_NAME).status() {
+ Ok(s) if !s.success() => println!("failure on exporting {ZPOOL_NAME}"),
+ _ => (),
+ }
+}
+
+fn mount_fs() -> Result<()> {
+ let iso_info = get_iso_info()?;
+ let product = iso_info.config.product;
+
+ println!("Activating VG '{product}'");
+ let res = Command::new("vgchange")
+ .arg("-ay")
+ .arg(product.to_string())
+ .output();
+ match res {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!(
+ "successfully activated VG '{product}': {}",
+ String::from_utf8(output.stdout)?
+ );
+ } else {
+ bail!(
+ "activation of VG '{product}' failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ match Command::new("mount")
+ .arg(format!("/dev/mapper/{product}-root"))
+ .arg("/target")
+ .output()
+ {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!("mounted root file system successfully");
+ } else {
+ bail!(
+ "mounting of root file system failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn umount_fs() -> Result<()> {
+ umount(TARGET_DIR)?;
+ Ok(())
+}
+
+fn mount_btrfs(btrfs_uuid: Option<String>) -> Result<()> {
+ let uuid = match btrfs_uuid {
+ Some(uuid) => uuid,
+ None => get_btrfs_uuid()?,
+ };
+
+ match Command::new("mount")
+ .arg("--uuid")
+ .arg(uuid)
+ .arg("/target")
+ .output()
+ {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!("mounted BTRFS root file system successfully");
+ } else {
+ bail!(
+ "mounting of BTRFS root file system failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn get_btrfs_uuid() -> Result<String> {
+ let output = Command::new("btrfs")
+ .arg("filesystem")
+ .arg("show")
+ .output()?;
+ if !output.status.success() {
+ bail!(
+ "Error checking for BTRFS file systems: {}",
+ String::from_utf8(output.stderr)?
+ );
+ }
+ let out = String::from_utf8(output.stdout)?;
+ let mut uuids = Vec::new();
+
+ let re_uuid =
+ Regex::new(r"uuid: ([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$")?;
+ for line in out.lines() {
+ if let Some(cap) = re_uuid.captures(line) {
+ if let Some(uuid) = cap.get(1) {
+ uuids.push(uuid.as_str());
+ }
+ }
+ }
+ match uuids.len() {
+ 0 => bail!("Could not find any BTRFS UUID"),
+ i if i > 1 => {
+ let uuid_list = uuids
+ .iter()
+ .fold(String::new(), |acc, &arg| format!("{acc}\n{arg}"));
+ bail!("Found {i} UUIDs:{uuid_list}\nPlease specify the UUID to use with the --btrfs-uuid parameter")
+ }
+ _ => (),
+ }
+ Ok(uuids[0].into())
+}
+
+fn bindmount() -> Result<()> {
+ println!("Bind mounting");
+ // https://github.com/nix-rust/nix/blob/7badbee1e388618457ed0d725c1091359f253012/test/test_mount.rs#L19
+ // https://github.com/nix-rust/nix/blob/7badbee1e388618457ed0d725c1091359f253012/test/test_mount.rs#L146
+ const NONE: Option<&'static [u8]> = None;
+
+ let flags = MsFlags::MS_BIND;
+ for item in BINDMOUNTS {
+ let source = path::Path::new("/").join(item);
+ let target = path::Path::new(TARGET_DIR).join(item);
+
+ println!("Bindmount {} to {}", source.display(), target.display());
+ mount(Some(source.as_path()), target.as_path(), NONE, flags, NONE)?;
+ }
+
+ let answer_path = path::Path::new("/mnt").join(ANSWER_MP);
+ if answer_path.exists() {
+ let target = path::Path::new(TARGET_DIR).join("mnt").join(ANSWER_MP);
+
+ println!("Create dir {}", target.display());
+ fs::create_dir_all(&target)?;
+
+ println!(
+ "Bindmount {} to {}",
+ answer_path.display(),
+ target.display()
+ );
+ mount(
+ Some(answer_path.as_path()),
+ target.as_path(),
+ NONE,
+ flags,
+ NONE,
+ )?;
+ }
+ Ok(())
+}
+
+fn bind_umount() -> Result<()> {
+ for item in BINDMOUNTS {
+ let target = path::Path::new(TARGET_DIR).join(item);
+ println!("Unmounting {}", target.display());
+ if let Err(e) = umount(target.as_path()) {
+ eprintln!("{e}");
+ }
+ }
+
+ let answer_target = path::Path::new(TARGET_DIR).join("mnt").join(ANSWER_MP);
+ if answer_target.exists() {
+ println!("Unmounting and removing answer mountpoint");
+ if let Err(e) = umount(answer_target.as_os_str()) {
+ eprintln!("{e}");
+ }
+ if let Err(e) = fs::remove_dir(answer_target) {
+ eprintln!("{e}");
+ }
+ }
+
+ Ok(())
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 31/36] auto-installer: answer: deny unknown fields
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (29 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 30/36] add proxmox-chroot utility Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 32/36] fetch-answer: move get_answer_file to utils Aaron Lauterer
` (4 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
This way, serde will throw errors if fields are not known.
This can help to reduce frustration if one might think to have set an
option, but for example a small type has happened.
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/answer.rs | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
index 94cebb3..57c2602 100644
--- a/proxmox-auto-installer/src/answer.rs
+++ b/proxmox-auto-installer/src/answer.rs
@@ -10,7 +10,7 @@ use std::{collections::BTreeMap, net::IpAddr};
/// storing them in a HashMap
#[derive(Clone, Deserialize, Debug)]
-#[serde(rename_all = "kebab-case")]
+#[serde(rename_all = "kebab-case", deny_unknown_fields)]
pub struct Answer {
pub global: Global,
pub network: Network,
@@ -19,6 +19,7 @@ pub struct Answer {
}
#[derive(Clone, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct Global {
pub country: String,
pub fqdn: Fqdn,
@@ -33,6 +34,7 @@ pub struct Global {
}
#[derive(Clone, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
struct NetworkInAnswer {
#[serde(default)]
pub use_dhcp: bool,
@@ -43,7 +45,7 @@ struct NetworkInAnswer {
}
#[derive(Clone, Deserialize, Debug)]
-#[serde(try_from = "NetworkInAnswer")]
+#[serde(try_from = "NetworkInAnswer", deny_unknown_fields)]
pub struct Network {
pub network_settings: NetworkSettings,
}
@@ -97,6 +99,7 @@ pub struct NetworkManual {
}
#[derive(Clone, Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
pub struct DiskSetup {
pub filesystem: Filesystem,
#[serde(default)]
@@ -109,7 +112,7 @@ pub struct DiskSetup {
}
#[derive(Clone, Debug, Deserialize)]
-#[serde(try_from = "DiskSetup")]
+#[serde(try_from = "DiskSetup", deny_unknown_fields)]
pub struct Disks {
pub fs_type: FsType,
pub disk_selection: DiskSelection,
@@ -207,14 +210,14 @@ pub enum DiskSelection {
Filter(BTreeMap<String, String>),
}
#[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)]
-#[serde(rename_all = "lowercase")]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
pub enum FilterMatch {
Any,
All,
}
#[derive(Clone, Deserialize, Serialize, Debug, PartialEq)]
-#[serde(rename_all = "lowercase")]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
pub enum Filesystem {
Ext4,
Xfs,
@@ -223,6 +226,7 @@ pub enum Filesystem {
}
#[derive(Clone, Copy, Default, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct ZfsOptions {
pub raid: Option<ZfsRaidLevel>,
pub ashift: Option<usize>,
@@ -234,6 +238,7 @@ pub struct ZfsOptions {
}
#[derive(Clone, Copy, Default, Deserialize, Serialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct LvmOptions {
pub hdsize: Option<f64>,
pub swapsize: Option<f64>,
@@ -243,6 +248,7 @@ pub struct LvmOptions {
}
#[derive(Clone, Copy, Default, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct BtrfsOptions {
pub hdsize: Option<f64>,
pub raid: Option<BtrfsRaidLevel>,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 32/36] fetch-answer: move get_answer_file to utils
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (30 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 31/36] auto-installer: answer: deny unknown fields Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 33/36] auto-installer: utils: define ISO specified settings Aaron Lauterer
` (3 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
and switch to accepting the full path to the answer file. This makes it
possible to use it in more situations than just the partition case.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/fetch_plugins/partition.rs | 23 +++++--------------
.../src/fetch_plugins/utils/mod.rs | 13 ++++++++++-
2 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/proxmox-fetch-answer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
index dbe5dda..2557c59 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -1,8 +1,8 @@
-use anyhow::{Error, Result};
+use anyhow::Result;
use log::info;
-use std::{fs::read_to_string, path::Path};
+use std::path::PathBuf;
-use crate::fetch_plugins::utils::mount_proxmoxinst_part;
+use crate::fetch_plugins::utils::{get_answer_file, mount_proxmoxinst_part};
static ANSWER_FILE: &str = "answer.toml";
@@ -12,21 +12,10 @@ impl FetchFromPartition {
/// Returns the contents of the answer file
pub fn get_answer() -> Result<String> {
info!("Checking for answer file on partition.");
- let mount_path = mount_proxmoxinst_part()?;
- let answer = Self::get_answer_file(&mount_path)?;
+ let mut mount_path = PathBuf::from(mount_proxmoxinst_part()?);
+ mount_path.push(ANSWER_FILE);
+ let answer = get_answer_file(&mount_path)?;
info!("Found answer file on partition.");
Ok(answer)
}
-
- /// Searches for answer file and returns contents if found
- fn get_answer_file(mount_path: &str) -> Result<String> {
- let answer_path = Path::new(mount_path).join(ANSWER_FILE);
- match answer_path.try_exists() {
- Ok(true) => Ok(read_to_string(answer_path)?),
- _ => Err(Error::msg(format!(
- "could not find answer file expected at: {}",
- answer_path.display()
- ))),
- }
- }
}
diff --git a/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
index e5ea4b8..29cb37d 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
@@ -1,7 +1,7 @@
use anyhow::{Error, Result};
use log::{info, warn};
use std::{
- fs::{self, create_dir_all},
+ fs::{self, create_dir_all, read_to_string},
path::{Path, PathBuf},
process::Command,
};
@@ -83,3 +83,14 @@ fn check_if_mounted(target_path: &str) -> Result<bool> {
}
Ok(false)
}
+
+/// Searches for answer file and returns contents if found
+pub fn get_answer_file(path: &PathBuf) -> Result<String> {
+ match path.try_exists() {
+ Ok(true) => Ok(read_to_string(path)?),
+ _ => Err(Error::msg(format!(
+ "could not find answer file expected at: {}",
+ path.display()
+ ))),
+ }
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 33/36] auto-installer: utils: define ISO specified settings
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (31 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 32/36] fetch-answer: move get_answer_file to utils Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 34/36] fetch-answer: use ISO specified configurations Aaron Lauterer
` (2 subsequent siblings)
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
These will be expected on the ISO itself and define the behavior of the
automated installation.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/utils.rs | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index ff90ae8..997ab34 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -1,4 +1,5 @@
use anyhow::{bail, Result};
+use clap::ValueEnum;
use glob::Pattern;
use log::info;
use std::{
@@ -14,7 +15,7 @@ use proxmox_installer_common::{
options::{FsType, NetworkOptions, ZfsChecksumOption, ZfsCompressOption},
setup::{InstallConfig, InstallZfsOption, LocaleInfo, RuntimeInfo, SetupInfo},
};
-use serde::Deserialize;
+use serde::{Deserialize, Serialize};
fn find_with_glob(pattern: &str, value: &str) -> Result<bool> {
let p = Pattern::new(pattern)?;
@@ -72,6 +73,23 @@ pub fn get_single_udev_index(
Ok(dev_index.unwrap())
}
+#[derive(Deserialize, Serialize, Debug, Clone, ValueEnum, PartialEq)]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
+pub enum AutoInstModes {
+ Auto,
+ Included,
+ Http,
+ Partition,
+}
+
+#[derive(Deserialize, Serialize, Debug)]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
+pub struct AutoInstSettings {
+ pub mode: AutoInstModes,
+ pub http_url: Option<String>,
+ pub cert_fingerprint: Option<String>,
+}
+
#[derive(Deserialize, Debug)]
struct IpLinksUdevInfo {
ifname: String,
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 34/36] fetch-answer: use ISO specified configurations
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (32 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 33/36] auto-installer: utils: define ISO specified settings Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 35/36] fetch-answer: dpcp: improve logging of steps taken Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand Aaron Lauterer
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
This patch switches the behavior to use the settings that can be
specified in the ISO.
This means, that it is possible to control how the answer file should be
fetched:
* auto - as usually, go through the options until one works (partition,
http)
* included - the answer file is included in the ISO
* partition - only check for an answer file in a partition called
'proxmoxinst' in lower or uppercase
* http - only fetch the answer file via an HTTP POST request.
Additionally it is possible to specify the HTTP URL directly in the ISO.
Placing the SSL fingerprint on a partition is not possible anymore. If
one wants to provide it right away (besides DHCP or DNS), it must be
incluced in the ISO itself. This reduced the need for another USB flash
drive.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-fetch-answer/Cargo.toml | 1 +
.../src/fetch_plugins/http.rs | 65 +++++++------------
proxmox-fetch-answer/src/main.rs | 64 ++++++++++++++----
3 files changed, 77 insertions(+), 53 deletions(-)
diff --git a/proxmox-fetch-answer/Cargo.toml b/proxmox-fetch-answer/Cargo.toml
index fbcca46..797c185 100644
--- a/proxmox-fetch-answer/Cargo.toml
+++ b/proxmox-fetch-answer/Cargo.toml
@@ -17,6 +17,7 @@ log = "0.4.20"
ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
rustls-native-certs = "0.6"
+toml = "0.7"
native-tls = "0.2"
sha2 = "0.10"
hex = "0.4"
diff --git a/proxmox-fetch-answer/src/fetch_plugins/http.rs b/proxmox-fetch-answer/src/fetch_plugins/http.rs
index 5772c42..4093131 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/http.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/http.rs
@@ -2,16 +2,12 @@ use anyhow::{bail, Error, Result};
use log::info;
use std::{
fs::{self, read_to_string},
- path::Path,
process::Command,
};
use crate::fetch_plugins::utils::post;
-use proxmox_auto_installer::sysinfo;
+use proxmox_auto_installer::{sysinfo, utils::AutoInstSettings};
-use super::utils;
-
-static CERT_FINGERPRINT_FILE: &str = "cert_fingerprint.txt";
static ANSWER_SUBDOMAIN: &str = "proxmoxinst";
static ANSWER_SUBDOMAIN_FP: &str = "proxmoxinst-fp";
@@ -37,30 +33,33 @@ pub struct FetchFromHTTP;
impl FetchFromHTTP {
/// Will try to fetch the answer.toml by sending a HTTP POST request. The URL can be configured
- /// either via DHCP or DNS.
- /// DHCP options are checked first. The SSL certificate need to be either trusted by the root
- /// certs or a SHA256 fingerprint needs to be provided. The SHA256 SSL fingerprint can either
- /// be placed in a `cert_fingerprint.txt` file in the `proxmoxinst` partition, as DHCP option,
- /// or as DNS TXT record. If provided, the `cert_fingerprint.txt` file has preference.
- pub fn get_answer() -> Result<String> {
- info!("Checking for certificate fingerprint in file.");
- let mut fingerprint: Option<String> = match Self::get_cert_fingerprint_from_file() {
- Ok(fp) => Some(fp),
- Err(err) => {
- info!("{err}");
- None
+ /// either via DHCP or DNS or preconfigured in the ISO.
+ /// If the URL is not defined in the ISO, it will first check DHCP options. The SSL certificate
+ /// needs to be either trusted by the root certs or a SHA256 fingerprint needs to be provided.
+ /// The SHA256 SSL fingerprint can either be defined in the ISO, as DHCP option, or as DNS TXT
+ /// record. If provided, the fingerprint provided in the ISO has preference.
+ pub fn get_answer(settings: &AutoInstSettings) -> Result<String> {
+ let mut fingerprint: Option<String> = match settings.cert_fingerprint.clone() {
+ Some(fp) => {
+ info!("SSL fingerprint provided through ISO.");
+ Some(fp)
}
+ None => None,
};
let answer_url: String;
-
- (answer_url, fingerprint) = match Self::fetch_dhcp(fingerprint.clone()) {
- Ok((url, fp)) => (url, fp),
- Err(err) => {
- info!("{err}");
- Self::fetch_dns(fingerprint.clone())?
- }
- };
+ if let Some(url) = settings.http_url.clone() {
+ info!("URL specified in ISO");
+ answer_url = url;
+ } else {
+ (answer_url, fingerprint) = match Self::fetch_dhcp(fingerprint.clone()) {
+ Ok((url, fp)) => (url, fp),
+ Err(err) => {
+ info!("{err}");
+ Self::fetch_dns(fingerprint.clone())?
+ }
+ };
+ }
if fingerprint.is_some() {
let fp = fingerprint.clone();
@@ -74,22 +73,6 @@ impl FetchFromHTTP {
Ok(answer)
}
- /// Reads certificate fingerprint from file
- pub fn get_cert_fingerprint_from_file() -> Result<String> {
- let mount_path = utils::mount_proxmoxinst_part()?;
- let cert_path = Path::new(mount_path.as_str()).join(CERT_FINGERPRINT_FILE);
- match cert_path.try_exists() {
- Ok(true) => {
- info!("Found certifacte fingerprint file.");
- Ok(fs::read_to_string(cert_path)?.trim().into())
- }
- _ => Err(Error::msg(format!(
- "could not find cert fingerprint file expected at: {}",
- cert_path.display()
- ))),
- }
- }
-
/// Fetches search domain from resolv.conf file
fn get_search_domain() -> Result<String> {
info!("Retrieving default search domain.");
diff --git a/proxmox-fetch-answer/src/main.rs b/proxmox-fetch-answer/src/main.rs
index 8c762e9..fe5d599 100644
--- a/proxmox-fetch-answer/src/main.rs
+++ b/proxmox-fetch-answer/src/main.rs
@@ -1,13 +1,15 @@
use anyhow::{anyhow, Error, Result};
use fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition};
use log::{error, info, LevelFilter};
-use proxmox_auto_installer::log::AutoInstLogger;
+use proxmox_auto_installer::{log::AutoInstLogger, utils::{AutoInstModes, AutoInstSettings}};
+use std::{fs, path::PathBuf};
use std::io::Write;
use std::process::{Command, ExitCode, Stdio};
mod fetch_plugins;
static LOGGER: AutoInstLogger = AutoInstLogger;
+static AUTOINST_MODE_FILE: &str = "/cdrom/autoinst-mode.toml";
pub fn init_log() -> Result<()> {
AutoInstLogger::init("/tmp/fetch_answer.log")?;
@@ -16,16 +18,40 @@ pub fn init_log() -> Result<()> {
.map_err(|err| anyhow!(err))
}
-fn fetch_answer() -> Result<String> {
- match FetchFromPartition::get_answer() {
- Ok(answer) => return Ok(answer),
- Err(err) => info!("Fetching answer file from partition failed: {err}"),
- }
- match FetchFromHTTP::get_answer() {
- Ok(answer) => return Ok(answer),
- Err(err) => info!("Fetching answer file via HTTP failed: {err}"),
- }
+fn fetch_answer(install_settings: &AutoInstSettings) -> Result<String> {
+ info!("Fetching answer file in mode {:?}:", &install_settings.mode);
+ match install_settings.mode {
+ AutoInstModes::Auto => {
+ match FetchFromPartition::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file from partition failed: {err}"),
+ }
+ match FetchFromHTTP::get_answer(install_settings) {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file via HTTP failed: {err}"),
+ }
+ },
+ AutoInstModes::Included => {
+ let answer_path = PathBuf::from("/cdrom/answer.toml");
+ match fetch_plugins::utils::get_answer_file(&answer_path) {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file from ISO failed: {err}"),
+ }
+ },
+ AutoInstModes::Partition => {
+ match FetchFromPartition::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file from partition failed: {err}"),
+ }
+ },
+ AutoInstModes::Http => {
+ match FetchFromHTTP::get_answer(install_settings) {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file via HTTP failed: {err}"),
+ }
+ },
+ }
Err(Error::msg("Could not find any answer file!"))
}
@@ -34,8 +60,22 @@ fn main() -> ExitCode {
panic!("could not initialize logging: {err}");
}
- info!("Fetching answer file");
- let answer = match fetch_answer() {
+ let raw_install_settings = match fs::read_to_string(AUTOINST_MODE_FILE) {
+ Ok(f) => f,
+ Err(err) => {
+ error!("Could not find needed file '{AUTOINST_MODE_FILE}' in live environment: {err}");
+ return ExitCode::FAILURE;
+ },
+ };
+ let install_settings: AutoInstSettings = match toml::from_str(raw_install_settings.as_str()) {
+ Ok(content) => content,
+ Err(err) => {
+ error!("Failed to parse '{AUTOINST_MODE_FILE}': {err}");
+ return ExitCode::FAILURE;
+ },
+ };
+
+ let answer = match fetch_answer(&install_settings) {
Ok(answer) => answer,
Err(err) => {
error!("Aborting: {}", err);
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 35/36] fetch-answer: dpcp: improve logging of steps taken
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (33 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 34/36] fetch-answer: use ISO specified configurations Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand Aaron Lauterer
35 siblings, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-fetch-answer/src/fetch_plugins/http.rs | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/proxmox-fetch-answer/src/fetch_plugins/http.rs b/proxmox-fetch-answer/src/fetch_plugins/http.rs
index 4093131..cd3775f 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/http.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/http.rs
@@ -142,6 +142,7 @@ impl FetchFromHTTP {
/// Tries to fetch answer URL and SSL fingerprint info from DHCP options
fn fetch_dhcp(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> {
+ info!("Checking DHCP options.");
let leases = fs::read_to_string(DHCP_LEASE_FILE)?;
let mut answer_url: Option<String> = None;
@@ -160,9 +161,16 @@ impl FetchFromHTTP {
let answer_url = match answer_url {
None => bail!("No DHCP option found for fetch URL."),
- Some(url) => url,
+ Some(url) => {
+ info!("Found URL for answer in DHCP option: '{url}'");
+ url
+ }
};
+ if let Some(fp) = fingerprint.clone() {
+ info!("Found SSL Fingerprint via DHCP: '{fp}'");
+ }
+
Ok((answer_url, fingerprint))
}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
` (34 preceding siblings ...)
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 35/36] fetch-answer: dpcp: improve logging of steps taken Aaron Lauterer
@ 2024-04-17 12:31 ` Aaron Lauterer
2024-04-18 8:48 ` Christoph Heiss
2024-04-18 11:38 ` [pve-devel] [PATCH installer v6 36/36 follow-up] " Aaron Lauterer
35 siblings, 2 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-17 12:31 UTC (permalink / raw)
To: pve-devel
This new subcommand makes it possible to prepare an ISO to use it for an
automated installation.
It is possible to control the behavior of the resulting automated ISO
with optional parameters.
If no target file is specified, the new ISO will be named with suffixes
to indicate it as automated and additional information. This should help
to distinct between the different options that were chosen to create it.
The code for parsing an answer file is moved to its own function.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-autoinst-helper/Cargo.toml | 1 +
proxmox-autoinst-helper/src/main.rs | 269 +++++++++++++++++++++++++---
2 files changed, 248 insertions(+), 22 deletions(-)
diff --git a/proxmox-autoinst-helper/Cargo.toml b/proxmox-autoinst-helper/Cargo.toml
index 2a88c0f..75399e0 100644
--- a/proxmox-autoinst-helper/Cargo.toml
+++ b/proxmox-autoinst-helper/Cargo.toml
@@ -19,3 +19,4 @@ serde_json = "1.0"
toml = "0.7"
log = "0.4.20"
regex = "1.7"
+which = "4.2.5"
diff --git a/proxmox-autoinst-helper/src/main.rs b/proxmox-autoinst-helper/src/main.rs
index fe1cbec..81bb54d 100644
--- a/proxmox-autoinst-helper/src/main.rs
+++ b/proxmox-autoinst-helper/src/main.rs
@@ -3,16 +3,29 @@ use clap::{Args, Parser, Subcommand, ValueEnum};
use glob::Pattern;
use regex::Regex;
use serde::Serialize;
-use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
+use std::{
+ collections::BTreeMap,
+ fs,
+ io::Read,
+ path::{Path, PathBuf},
+ process::{Command, Stdio},
+};
+use which::which;
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
sysinfo,
- utils::{get_matched_udev_indexes, get_nic_list, get_single_udev_index},
+ utils::{
+ get_matched_udev_indexes, get_nic_list, get_single_udev_index, AutoInstModes,
+ AutoInstSettings,
+ },
};
-/// This tool validates the format of an answer file. Additionally it can test match filters and
+static PROXMOX_ISO_FLAG: &str = "/autoinst-capable";
+
+/// This tool can be used to prepare a Proxmox installation ISO for automated installations.
+/// Additional uses are to validate the format of an answer file or to test match filters and
/// print information on the properties to match against for the current hardware.
#[derive(Parser, Debug)]
#[command(author, version, about, long_about = None)]
@@ -23,6 +36,7 @@ struct Cli {
#[derive(Subcommand, Debug)]
enum Commands {
+ PrepareIso(CommandPrepareISO),
ValidateAnswer(CommandValidateAnswer),
DeviceMatch(CommandDeviceMatch),
DeviceInfo(CommandDeviceInfo),
@@ -76,6 +90,62 @@ struct CommandValidateAnswer {
debug: bool,
}
+/// Prepare an ISO for automated installation.
+///
+/// The final ISO will try to fetch an answer file automatically. It will first search for a
+/// partition / file-system called "PROXMOXINST" (or lowercase) and a file in the root named
+/// "answer.toml".
+///
+/// If that is not found, it will try to fetch an answer file via an HTTP Post request. The URL for
+/// it can be defined for the ISO with the '--url', '-u' argument. If not present, it will try to
+/// get a URL from a DHCP option (250, TXT) or as a DNS TXT record at 'proxmoxinst.{search
+/// domain}'.
+///
+/// The TLS certificate fingerprint can either be defined via the '--cert-fingerprint', '-c'
+/// argument or alternatively via the custom DHCP option (251, TXT) or in a DNS TXT record located
+/// at 'proxmoxinst-fp.{search domain}'.
+///
+/// The latter options to provide the TLS fingerprint will only be used if the same method was used
+/// to retrieve the URL. For example, the DNS TXT record for the fingerprint will only be used, if
+/// no one was configured with the '--cert-fingerprint' parameter and if the URL was retrieved via
+/// the DNS TXT record.
+///
+/// The behavior of how to fetch an answer file can be overridden with the '--install-mode', '-i'
+/// parameter. The answer file can be{n}
+/// * integrated into the ISO itself ('included'){n}
+/// * needs to be present in a partition / file-system called 'PROXMOXINST' ('partition'){n}
+/// * only be requested via an HTTP Post request ('http').
+#[derive(Args, Debug)]
+struct CommandPrepareISO {
+ /// Path to the source ISO
+ source: PathBuf,
+
+ /// Path to store the final ISO to.
+ #[arg(short, long)]
+ target: Option<PathBuf>,
+
+ /// Where to fetch the answer file from.
+ #[arg(short, long, value_enum, default_value_t=AutoInstModes::Auto)]
+ install_mode: AutoInstModes,
+
+ /// Include the specified answer file in the ISO. Requires the '--install-mode', '-i' parameter
+ /// to be set to 'included'.
+ #[arg(short, long)]
+ answer_file: Option<PathBuf>,
+
+ /// Specify URL for fetching the answer file via HTTP
+ #[arg(short, long)]
+ url: Option<String>,
+
+ /// Pin the ISO to the specified SHA256 TLS certificate fingerprint.
+ #[arg(short, long)]
+ cert_fingerprint: Option<String>,
+
+ /// Tmp directory to use.
+ #[arg(long)]
+ tmp: Option<String>,
+}
+
/// Show identifiers for the current machine. This information is part of the POST request to fetch
/// an answer file.
#[derive(Args, Debug)]
@@ -116,6 +186,7 @@ struct Devs {
fn main() {
let args = Cli::parse();
let res = match &args.command {
+ Commands::PrepareIso(args) => prepare_iso(args),
Commands::ValidateAnswer(args) => validate_answer(args),
Commands::DeviceInfo(args) => info(args),
Commands::DeviceMatch(args) => match_filter(args),
@@ -188,25 +259,7 @@ fn match_filter(args: &CommandDeviceMatch) -> Result<()> {
}
fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
- let mut file = match fs::File::open(&args.path) {
- Ok(file) => file,
- Err(err) => bail!(
- "Opening answer file '{}' failed: {err}",
- args.path.display()
- ),
- };
- let mut contents = String::new();
- if let Err(err) = file.read_to_string(&mut contents) {
- bail!("Reading from file '{}' failed: {err}", args.path.display());
- }
-
- let answer: Answer = match toml::from_str(&contents) {
- Ok(answer) => {
- println!("The file was parsed successfully, no syntax errors found!");
- answer
- }
- Err(err) => bail!("Error parsing answer file: {err}"),
- };
+ let answer = parse_answer(&args.path)?;
if args.debug {
println!("Parsed data from answer file:\n{:#?}", answer);
}
@@ -221,6 +274,128 @@ fn show_identifiers(_args: &CommandIdentifiers) -> Result<()> {
Ok(())
}
+fn prepare_iso(args: &CommandPrepareISO) -> Result<()> {
+ check_prepare_requirements(args)?;
+
+ if args.install_mode == AutoInstModes::Included && args.answer_file.is_none() {
+ bail!("Missing path to answer file needed for 'direct' install mode.");
+ }
+ if args.install_mode == AutoInstModes::Included && args.cert_fingerprint.is_some() {
+ bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
+ }
+ if args.install_mode == AutoInstModes::Included && args.url.is_some() {
+ bail!("No URL needed for direct install mode. Drop the parameter!");
+ }
+ if args.answer_file.is_some() && args.install_mode != AutoInstModes::Included {
+ bail!("Set '-i', '--install-mode' to 'included' to place the answer file directly in the ISO.");
+ }
+ if args.install_mode == AutoInstModes::Partition && args.cert_fingerprint.is_some() {
+ bail!("No certificate fingerprint needed for partition install mode. Drop the parameter!");
+ }
+ if args.install_mode == AutoInstModes::Partition && args.url.is_some() {
+ bail!("No URL needed for partition install mode. Drop the parameter!");
+ }
+
+ if let Some(file) = &args.answer_file {
+ println!("Checking provided answer file...");
+ parse_answer(file)?;
+ }
+
+ let mut tmp_base = PathBuf::new();
+ if args.tmp.is_some() {
+ tmp_base.push(args.tmp.as_ref().unwrap());
+ } else {
+ tmp_base.push(args.source.parent().unwrap());
+ tmp_base.push(".proxmox-iso-prepare");
+ }
+ fs::create_dir_all(&tmp_base)?;
+
+ let mut tmp_iso = tmp_base.clone();
+ tmp_iso.push("proxmox.iso");
+ let mut tmp_answer = tmp_base.clone();
+ tmp_answer.push("answer.toml");
+
+ println!("Copying source ISO to temporary location...");
+ fs::copy(&args.source, &tmp_iso)?;
+ println!("Done copying source ISO");
+
+ println!("Preparing ISO...");
+ let install_mode = AutoInstSettings {
+ mode: args.install_mode.clone(),
+ http_url: args.url.clone(),
+ cert_fingerprint: args.cert_fingerprint.clone(),
+ };
+ let mut instmode_file_tmp = tmp_base.clone();
+ instmode_file_tmp.push("autoinst-mode.toml");
+ fs::write(&instmode_file_tmp, toml::to_string_pretty(&install_mode)?)?;
+
+ inject_file_to_iso(&tmp_iso, &instmode_file_tmp, "/autoinst-mode.toml")?;
+
+ if let Some(answer) = &args.answer_file {
+ fs::copy(answer, &tmp_answer)?;
+ inject_file_to_iso(&tmp_iso, &tmp_answer, "/answer.toml")?;
+ }
+
+ println!("Done preparing iso.");
+ println!("Move ISO to target location...");
+ let iso_target = final_iso_location(args);
+ fs::rename(&tmp_iso, &iso_target)?;
+ println!("Cleaning up...");
+ fs::remove_dir_all(&tmp_base)?;
+ println!("Final ISO is available at {}.", &iso_target.display());
+
+ Ok(())
+}
+
+fn final_iso_location(args: &CommandPrepareISO) -> PathBuf {
+ if let Some(specified) = args.target.clone() {
+ return specified;
+ }
+ let mut suffix: String = match args.install_mode {
+ AutoInstModes::Auto => "auto".into(),
+ AutoInstModes::Http => "auto-http".into(),
+ AutoInstModes::Included => "auto-answer-included".into(),
+ AutoInstModes::Partition => "auto-part".into(),
+ };
+
+ if args.url.is_some() {
+ suffix.push_str("-url");
+ }
+ if args.cert_fingerprint.is_some() {
+ suffix.push_str("-fp");
+ }
+
+ let base = args.source.parent().unwrap();
+ let iso = args.source.file_stem().unwrap();
+
+ let mut target = base.to_path_buf();
+ target.push(format!("{}-{}.iso", iso.to_str().unwrap(), suffix));
+
+ target.to_path_buf()
+}
+
+fn inject_file_to_iso(iso: &PathBuf, file: &PathBuf, location: &str) -> Result<()> {
+ let result = Command::new("xorriso")
+ .arg("--boot_image")
+ .arg("any")
+ .arg("keep")
+ .arg("-dev")
+ .arg(iso)
+ .arg("-map")
+ .arg(file)
+ .arg(location)
+ .output()?;
+ if !result.status.success() {
+ bail!(
+ "Error injecting {} into {}: {}",
+ file.display(),
+ iso.display(),
+ String::from_utf8(result.stderr)?
+ );
+ }
+ Ok(())
+}
+
fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let unwantend_block_devs = vec![
"ram[0-9]*",
@@ -335,3 +510,53 @@ fn get_udev_properties(path: &PathBuf) -> Result<String> {
}
Ok(String::from_utf8(udev_output.stdout)?)
}
+
+fn parse_answer(path: &PathBuf) -> Result<Answer> {
+ let mut file = match fs::File::open(path) {
+ Ok(file) => file,
+ Err(err) => bail!("Opening answer file '{}' failed: {err}", path.display()),
+ };
+ let mut contents = String::new();
+ if let Err(err) = file.read_to_string(&mut contents) {
+ bail!("Reading from file '{}' failed: {err}", path.display());
+ }
+ match toml::from_str(&contents) {
+ Ok(answer) => {
+ println!("The file was parsed successfully, no syntax errors found!");
+ Ok(answer)
+ }
+ Err(err) => bail!("Error parsing answer file: {err}"),
+ }
+}
+
+fn check_prepare_requirements(args: &CommandPrepareISO) -> Result<()> {
+ match which("xorriso") {
+ Ok(_) => (),
+ Err(_) => bail!("Could not find 'xorriso'. Please install it and try again"),
+ }
+
+ match Path::try_exists(&args.source) {
+ Ok(true) => (),
+ Ok(false) => bail!("Source file does not exist."),
+ Err(_) => bail!("Source file does not exist."),
+ }
+
+ match Command::new("xorriso")
+ .arg("-dev")
+ .arg(&args.source)
+ .arg("-find")
+ .arg(PROXMOX_ISO_FLAG)
+ .stderr(Stdio::null())
+ .stdout(Stdio::null())
+ .status()
+ {
+ Ok(v) => {
+ if !v.success() {
+ bail!("The source ISO file is not able to be installed automatically. Please try a more current one.");
+ }
+ }
+ Err(_) => bail!("Could not run 'xorriso'. Please install it."),
+ };
+
+ Ok(())
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand Aaron Lauterer
@ 2024-04-18 8:48 ` Christoph Heiss
2024-04-18 9:13 ` Aaron Lauterer
2024-04-18 13:39 ` Thomas Lamprecht
2024-04-18 11:38 ` [pve-devel] [PATCH installer v6 36/36 follow-up] " Aaron Lauterer
1 sibling, 2 replies; 41+ messages in thread
From: Christoph Heiss @ 2024-04-18 8:48 UTC (permalink / raw)
To: Aaron Lauterer; +Cc: Proxmox VE development discussion
Just quick three notes inline; nits other than the crate thing.
Did not review in depth, LGTM overall tho.
On Wed, Apr 17, 2024 at 02:31:08PM +0200, Aaron Lauterer wrote:
[..]
> diff --git a/proxmox-autoinst-helper/Cargo.toml b/proxmox-autoinst-helper/Cargo.toml
> index 2a88c0f..75399e0 100644
> --- a/proxmox-autoinst-helper/Cargo.toml
> +++ b/proxmox-autoinst-helper/Cargo.toml
> @@ -19,3 +19,4 @@ serde_json = "1.0"
> toml = "0.7"
> log = "0.4.20"
> regex = "1.7"
> +which = "4.2.5"
Misses the debian/control entry, but see also below.
[..]
>
> +fn prepare_iso(args: &CommandPrepareISO) -> Result<()> {
> + check_prepare_requirements(args)?;
> +
> + if args.install_mode == AutoInstModes::Included && args.answer_file.is_none() {
> + bail!("Missing path to answer file needed for 'direct' install mode.");
> + }
> + if args.install_mode == AutoInstModes::Included && args.cert_fingerprint.is_some() {
> + bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
> + }
> + if args.install_mode == AutoInstModes::Included && args.url.is_some() {
> + bail!("No URL needed for direct install mode. Drop the parameter!");
> + }
if args.install_mode == AutoInstModes::Included {
if args.answer_file.is_none() {
bail!("Missing path to answer file needed for 'direct' install mode.");
}
if args.cert_fingerprint.is_some() {
bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
}
if args.url.is_some() {
bail!("No URL needed for direct install mode. Drop the parameter!");
}
} else if (args.install_mode == AutoInstModes::Partition) { .. }
.. maybe, to avoid the repeated condition? (The resulting visual
grouping is also nice)
> + if args.answer_file.is_some() && args.install_mode != AutoInstModes::Included {
> + bail!("Set '-i', '--install-mode' to 'included' to place the answer file directly in the ISO.");
> + }
> + if args.install_mode == AutoInstModes::Partition && args.cert_fingerprint.is_some() {
> + bail!("No certificate fingerprint needed for partition install mode. Drop the parameter!");
> + }
> + if args.install_mode == AutoInstModes::Partition && args.url.is_some() {
> + bail!("No URL needed for partition install mode. Drop the parameter!");
> + }
> +
[..]
> +
> fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
> let unwantend_block_devs = vec![
> "ram[0-9]*",
> @@ -335,3 +510,53 @@ fn get_udev_properties(path: &PathBuf) -> Result<String> {
> }
> Ok(String::from_utf8(udev_output.stdout)?)
> }
> +
> +fn parse_answer(path: &PathBuf) -> Result<Answer> {
> + let mut file = match fs::File::open(path) {
> + Ok(file) => file,
> + Err(err) => bail!("Opening answer file '{}' failed: {err}", path.display()),
> + };
> + let mut contents = String::new();
> + if let Err(err) = file.read_to_string(&mut contents) {
> + bail!("Reading from file '{}' failed: {err}", path.display());
> + }
There is also std::fs::read_to_string() for exactly that; and would
avoid the whole open/close dance :^)
(Seems I missed that when reviewing the patch that introduced
validate_answer())
> + match toml::from_str(&contents) {
> + Ok(answer) => {
> + println!("The file was parsed successfully, no syntax errors found!");
> + Ok(answer)
> + }
> + Err(err) => bail!("Error parsing answer file: {err}"),
> + }
> +}
> +
> +fn check_prepare_requirements(args: &CommandPrepareISO) -> Result<()> {
> + match which("xorriso") {
Do we really need _yet another_ crate dependency for that? Below is a
check / bail! anyway when running the command proper.
And if we really want a explicit check beforehand, I'd just do something
like
fn which(name: &str) -> Result<()> {
match Command::new(name).output() {
Ok(_) => Ok(()),
Err(err) => Err(err.into()),
}
}
> + Ok(_) => (),
> + Err(_) => bail!("Could not find 'xorriso'. Please install it and try again"),
> + }
> +
> + match Path::try_exists(&args.source) {
> + Ok(true) => (),
> + Ok(false) => bail!("Source file does not exist."),
> + Err(_) => bail!("Source file does not exist."),
> + }
> +
> + match Command::new("xorriso")
> + .arg("-dev")
> + .arg(&args.source)
> + .arg("-find")
> + .arg(PROXMOX_ISO_FLAG)
> + .stderr(Stdio::null())
> + .stdout(Stdio::null())
> + .status()
> + {
> + Ok(v) => {
> + if !v.success() {
> + bail!("The source ISO file is not able to be installed automatically. Please try a more current one.");
> + }
> + }
> + Err(_) => bail!("Could not run 'xorriso'. Please install it."),
> + };
> +
> + Ok(())
> +}
> --
> 2.39.2
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand
2024-04-18 8:48 ` Christoph Heiss
@ 2024-04-18 9:13 ` Aaron Lauterer
2024-04-18 13:39 ` Thomas Lamprecht
1 sibling, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-18 9:13 UTC (permalink / raw)
To: Christoph Heiss; +Cc: Proxmox VE development discussion
On 2024-04-18 10:48, Christoph Heiss wrote:
> Just quick three notes inline; nits other than the crate thing.
> Did not review in depth, LGTM overall tho.
>
> On Wed, Apr 17, 2024 at 02:31:08PM +0200, Aaron Lauterer wrote:
> [..]
>> diff --git a/proxmox-autoinst-helper/Cargo.toml b/proxmox-autoinst-helper/Cargo.toml
>> index 2a88c0f..75399e0 100644
>> --- a/proxmox-autoinst-helper/Cargo.toml
>> +++ b/proxmox-autoinst-helper/Cargo.toml
>> @@ -19,3 +19,4 @@ serde_json = "1.0"
>> toml = "0.7"
>> log = "0.4.20"
>> regex = "1.7"
>> +which = "4.2.5"
>
> Misses the debian/control entry, but see also below.
>
thanks for catching that, more further down
> [..]
>>
>> +fn prepare_iso(args: &CommandPrepareISO) -> Result<()> {
>> + check_prepare_requirements(args)?;
>> +
>> + if args.install_mode == AutoInstModes::Included && args.answer_file.is_none() {
>> + bail!("Missing path to answer file needed for 'direct' install mode.");
>> + }
>> + if args.install_mode == AutoInstModes::Included && args.cert_fingerprint.is_some() {
>> + bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
>> + }
>> + if args.install_mode == AutoInstModes::Included && args.url.is_some() {
>> + bail!("No URL needed for direct install mode. Drop the parameter!");
>> + }
>
> if args.install_mode == AutoInstModes::Included {
> if args.answer_file.is_none() {
> bail!("Missing path to answer file needed for 'direct' install mode.");
> }
> if args.cert_fingerprint.is_some() {
> bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
> }
> if args.url.is_some() {
> bail!("No URL needed for direct install mode. Drop the parameter!");
> }
> } else if (args.install_mode == AutoInstModes::Partition) { .. }
>
> .. maybe, to avoid the repeated condition? (The resulting visual
> grouping is also nice)
good points
>
>
>> + if args.answer_file.is_some() && args.install_mode != AutoInstModes::Included {
>> + bail!("Set '-i', '--install-mode' to 'included' to place the answer file directly in the ISO.");
>> + }
>> + if args.install_mode == AutoInstModes::Partition && args.cert_fingerprint.is_some() {
>> + bail!("No certificate fingerprint needed for partition install mode. Drop the parameter!");
>> + }
>> + if args.install_mode == AutoInstModes::Partition && args.url.is_some() {
>> + bail!("No URL needed for partition install mode. Drop the parameter!");
>> + }
>> +
> [..]
>> +
>> fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
>> let unwantend_block_devs = vec![
>> "ram[0-9]*",
>> @@ -335,3 +510,53 @@ fn get_udev_properties(path: &PathBuf) -> Result<String> {
>> }
>> Ok(String::from_utf8(udev_output.stdout)?)
>> }
>> +
>> +fn parse_answer(path: &PathBuf) -> Result<Answer> {
>> + let mut file = match fs::File::open(path) {
>> + Ok(file) => file,
>> + Err(err) => bail!("Opening answer file '{}' failed: {err}", path.display()),
>> + };
>> + let mut contents = String::new();
>> + if let Err(err) = file.read_to_string(&mut contents) {
>> + bail!("Reading from file '{}' failed: {err}", path.display());
>> + }
>
> There is also std::fs::read_to_string() for exactly that; and would
> avoid the whole open/close dance :^)
>
> (Seems I missed that when reviewing the patch that introduced
> validate_answer())
ah yeah, I can change that in a follow up
>
>> + match toml::from_str(&contents) {
>> + Ok(answer) => {
>> + println!("The file was parsed successfully, no syntax errors found!");
>> + Ok(answer)
>> + }
>> + Err(err) => bail!("Error parsing answer file: {err}"),
>> + }
>> +}
>> +
>> +fn check_prepare_requirements(args: &CommandPrepareISO) -> Result<()> {
>> + match which("xorriso") {
>
> Do we really need _yet another_ crate dependency for that? Below is a
> check / bail! anyway when running the command proper.
>
> And if we really want a explicit check beforehand, I'd just do something
> like
>
> fn which(name: &str) -> Result<()> {
> match Command::new(name).output() {
> Ok(_) => Ok(()),
> Err(err) => Err(err.into()),
> }
> }
I really would like to have an explicit check and nice warning if it
isn't present to reduce friction for users.
Initially, I thought of doing something like that, but considered it a
bit too hacky.
But the "do we want an additional crate" argument could be reason enough
to switch it over to a test like this.
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* [pve-devel] [PATCH installer v6 36/36 follow-up] autoinst-helper: add prepare-iso subcommand
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand Aaron Lauterer
2024-04-18 8:48 ` Christoph Heiss
@ 2024-04-18 11:38 ` Aaron Lauterer
1 sibling, 0 replies; 41+ messages in thread
From: Aaron Lauterer @ 2024-04-18 11:38 UTC (permalink / raw)
To: pve-devel
This new subcommand makes it possible to prepare an ISO to use it for an
automated installation.
It is possible to control the behavior of the resulting automated ISO
with optional parameters.
If no target file is specified, the new ISO will be named with suffixes
to indicate it as automated and additional information. This should help
to distinct between the different options that were chosen to create it.
The code for parsing an answer file is moved to its own function.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
changes: incorporated feedback from @cheiss (thx!)
* reorder parameter condition checks
* dropped use of 'which' crate
* dropped explicit check for 'xorriso' as for now, it will be run (and
fail if not present) before we do any expensive operation like copying
the source ISO.
proxmox-autoinst-helper/src/main.rs | 268 +++++++++++++++++++++++++---
1 file changed, 246 insertions(+), 22 deletions(-)
diff --git a/proxmox-autoinst-helper/src/main.rs b/proxmox-autoinst-helper/src/main.rs
index fe1cbec..9b04a24 100644
--- a/proxmox-autoinst-helper/src/main.rs
+++ b/proxmox-autoinst-helper/src/main.rs
@@ -3,16 +3,28 @@ use clap::{Args, Parser, Subcommand, ValueEnum};
use glob::Pattern;
use regex::Regex;
use serde::Serialize;
-use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
+use std::{
+ collections::BTreeMap,
+ fs,
+ io::Read,
+ path::{Path, PathBuf},
+ process::{Command, Stdio},
+};
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
sysinfo,
- utils::{get_matched_udev_indexes, get_nic_list, get_single_udev_index},
+ utils::{
+ get_matched_udev_indexes, get_nic_list, get_single_udev_index, AutoInstModes,
+ AutoInstSettings,
+ },
};
-/// This tool validates the format of an answer file. Additionally it can test match filters and
+static PROXMOX_ISO_FLAG: &str = "/autoinst-capable";
+
+/// This tool can be used to prepare a Proxmox installation ISO for automated installations.
+/// Additional uses are to validate the format of an answer file or to test match filters and
/// print information on the properties to match against for the current hardware.
#[derive(Parser, Debug)]
#[command(author, version, about, long_about = None)]
@@ -23,6 +35,7 @@ struct Cli {
#[derive(Subcommand, Debug)]
enum Commands {
+ PrepareIso(CommandPrepareISO),
ValidateAnswer(CommandValidateAnswer),
DeviceMatch(CommandDeviceMatch),
DeviceInfo(CommandDeviceInfo),
@@ -76,6 +89,62 @@ struct CommandValidateAnswer {
debug: bool,
}
+/// Prepare an ISO for automated installation.
+///
+/// The final ISO will try to fetch an answer file automatically. It will first search for a
+/// partition / file-system called "PROXMOXINST" (or lowercase) and a file in the root named
+/// "answer.toml".
+///
+/// If that is not found, it will try to fetch an answer file via an HTTP Post request. The URL for
+/// it can be defined for the ISO with the '--url', '-u' argument. If not present, it will try to
+/// get a URL from a DHCP option (250, TXT) or as a DNS TXT record at 'proxmoxinst.{search
+/// domain}'.
+///
+/// The TLS certificate fingerprint can either be defined via the '--cert-fingerprint', '-c'
+/// argument or alternatively via the custom DHCP option (251, TXT) or in a DNS TXT record located
+/// at 'proxmoxinst-fp.{search domain}'.
+///
+/// The latter options to provide the TLS fingerprint will only be used if the same method was used
+/// to retrieve the URL. For example, the DNS TXT record for the fingerprint will only be used, if
+/// no one was configured with the '--cert-fingerprint' parameter and if the URL was retrieved via
+/// the DNS TXT record.
+///
+/// The behavior of how to fetch an answer file can be overridden with the '--install-mode', '-i'
+/// parameter. The answer file can be{n}
+/// * integrated into the ISO itself ('included'){n}
+/// * needs to be present in a partition / file-system called 'PROXMOXINST' ('partition'){n}
+/// * only be requested via an HTTP Post request ('http').
+#[derive(Args, Debug)]
+struct CommandPrepareISO {
+ /// Path to the source ISO
+ source: PathBuf,
+
+ /// Path to store the final ISO to.
+ #[arg(short, long)]
+ target: Option<PathBuf>,
+
+ /// Where to fetch the answer file from.
+ #[arg(short, long, value_enum, default_value_t=AutoInstModes::Auto)]
+ install_mode: AutoInstModes,
+
+ /// Include the specified answer file in the ISO. Requires the '--install-mode', '-i' parameter
+ /// to be set to 'included'.
+ #[arg(short, long)]
+ answer_file: Option<PathBuf>,
+
+ /// Specify URL for fetching the answer file via HTTP
+ #[arg(short, long)]
+ url: Option<String>,
+
+ /// Pin the ISO to the specified SHA256 TLS certificate fingerprint.
+ #[arg(short, long)]
+ cert_fingerprint: Option<String>,
+
+ /// Tmp directory to use.
+ #[arg(long)]
+ tmp: Option<String>,
+}
+
/// Show identifiers for the current machine. This information is part of the POST request to fetch
/// an answer file.
#[derive(Args, Debug)]
@@ -116,6 +185,7 @@ struct Devs {
fn main() {
let args = Cli::parse();
let res = match &args.command {
+ Commands::PrepareIso(args) => prepare_iso(args),
Commands::ValidateAnswer(args) => validate_answer(args),
Commands::DeviceInfo(args) => info(args),
Commands::DeviceMatch(args) => match_filter(args),
@@ -188,25 +258,7 @@ fn match_filter(args: &CommandDeviceMatch) -> Result<()> {
}
fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
- let mut file = match fs::File::open(&args.path) {
- Ok(file) => file,
- Err(err) => bail!(
- "Opening answer file '{}' failed: {err}",
- args.path.display()
- ),
- };
- let mut contents = String::new();
- if let Err(err) = file.read_to_string(&mut contents) {
- bail!("Reading from file '{}' failed: {err}", args.path.display());
- }
-
- let answer: Answer = match toml::from_str(&contents) {
- Ok(answer) => {
- println!("The file was parsed successfully, no syntax errors found!");
- answer
- }
- Err(err) => bail!("Error parsing answer file: {err}"),
- };
+ let answer = parse_answer(&args.path)?;
if args.debug {
println!("Parsed data from answer file:\n{:#?}", answer);
}
@@ -221,6 +273,133 @@ fn show_identifiers(_args: &CommandIdentifiers) -> Result<()> {
Ok(())
}
+fn prepare_iso(args: &CommandPrepareISO) -> Result<()> {
+ check_prepare_requirements(args)?;
+
+ if args.install_mode == AutoInstModes::Included {
+ if args.answer_file.is_none() {
+ bail!("Missing path to answer file needed for 'direct' install mode.");
+ }
+ if args.cert_fingerprint.is_some() {
+ bail!("No certificate fingerprint needed for direct install mode. Drop the parameter!");
+ }
+ if args.url.is_some() {
+ bail!("No URL needed for direct install mode. Drop the parameter!");
+ }
+ } else if args.install_mode == AutoInstModes::Partition {
+ if args.cert_fingerprint.is_some() {
+ bail!(
+ "No certificate fingerprint needed for partition install mode. Drop the parameter!"
+ );
+ }
+ if args.url.is_some() {
+ bail!("No URL needed for partition install mode. Drop the parameter!");
+ }
+ }
+ if args.answer_file.is_some() && args.install_mode != AutoInstModes::Included {
+ bail!("Set '-i', '--install-mode' to 'included' to place the answer file directly in the ISO.");
+ }
+
+ if let Some(file) = &args.answer_file {
+ println!("Checking provided answer file...");
+ parse_answer(file)?;
+ }
+
+ let mut tmp_base = PathBuf::new();
+ if args.tmp.is_some() {
+ tmp_base.push(args.tmp.as_ref().unwrap());
+ } else {
+ tmp_base.push(args.source.parent().unwrap());
+ tmp_base.push(".proxmox-iso-prepare");
+ }
+ fs::create_dir_all(&tmp_base)?;
+
+ let mut tmp_iso = tmp_base.clone();
+ tmp_iso.push("proxmox.iso");
+ let mut tmp_answer = tmp_base.clone();
+ tmp_answer.push("answer.toml");
+
+ println!("Copying source ISO to temporary location...");
+ fs::copy(&args.source, &tmp_iso)?;
+ println!("Done copying source ISO");
+
+ println!("Preparing ISO...");
+ let install_mode = AutoInstSettings {
+ mode: args.install_mode.clone(),
+ http_url: args.url.clone(),
+ cert_fingerprint: args.cert_fingerprint.clone(),
+ };
+ let mut instmode_file_tmp = tmp_base.clone();
+ instmode_file_tmp.push("autoinst-mode.toml");
+ fs::write(&instmode_file_tmp, toml::to_string_pretty(&install_mode)?)?;
+
+ inject_file_to_iso(&tmp_iso, &instmode_file_tmp, "/autoinst-mode.toml")?;
+
+ if let Some(answer) = &args.answer_file {
+ fs::copy(answer, &tmp_answer)?;
+ inject_file_to_iso(&tmp_iso, &tmp_answer, "/answer.toml")?;
+ }
+
+ println!("Done preparing iso.");
+ println!("Move ISO to target location...");
+ let iso_target = final_iso_location(args);
+ fs::rename(&tmp_iso, &iso_target)?;
+ println!("Cleaning up...");
+ fs::remove_dir_all(&tmp_base)?;
+ println!("Final ISO is available at {}.", &iso_target.display());
+
+ Ok(())
+}
+
+fn final_iso_location(args: &CommandPrepareISO) -> PathBuf {
+ if let Some(specified) = args.target.clone() {
+ return specified;
+ }
+ let mut suffix: String = match args.install_mode {
+ AutoInstModes::Auto => "auto".into(),
+ AutoInstModes::Http => "auto-http".into(),
+ AutoInstModes::Included => "auto-answer-included".into(),
+ AutoInstModes::Partition => "auto-part".into(),
+ };
+
+ if args.url.is_some() {
+ suffix.push_str("-url");
+ }
+ if args.cert_fingerprint.is_some() {
+ suffix.push_str("-fp");
+ }
+
+ let base = args.source.parent().unwrap();
+ let iso = args.source.file_stem().unwrap();
+
+ let mut target = base.to_path_buf();
+ target.push(format!("{}-{}.iso", iso.to_str().unwrap(), suffix));
+
+ target.to_path_buf()
+}
+
+fn inject_file_to_iso(iso: &PathBuf, file: &PathBuf, location: &str) -> Result<()> {
+ let result = Command::new("xorriso")
+ .arg("--boot_image")
+ .arg("any")
+ .arg("keep")
+ .arg("-dev")
+ .arg(iso)
+ .arg("-map")
+ .arg(file)
+ .arg(location)
+ .output()?;
+ if !result.status.success() {
+ bail!(
+ "Error injecting {} into {}: {}",
+ file.display(),
+ iso.display(),
+ String::from_utf8(result.stderr)?
+ );
+ }
+ Ok(())
+}
+
fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let unwantend_block_devs = vec![
"ram[0-9]*",
@@ -335,3 +514,48 @@ fn get_udev_properties(path: &PathBuf) -> Result<String> {
}
Ok(String::from_utf8(udev_output.stdout)?)
}
+
+fn parse_answer(path: &PathBuf) -> Result<Answer> {
+ let mut file = match fs::File::open(path) {
+ Ok(file) => file,
+ Err(err) => bail!("Opening answer file '{}' failed: {err}", path.display()),
+ };
+ let mut contents = String::new();
+ if let Err(err) = file.read_to_string(&mut contents) {
+ bail!("Reading from file '{}' failed: {err}", path.display());
+ }
+ match toml::from_str(&contents) {
+ Ok(answer) => {
+ println!("The file was parsed successfully, no syntax errors found!");
+ Ok(answer)
+ }
+ Err(err) => bail!("Error parsing answer file: {err}"),
+ }
+}
+
+fn check_prepare_requirements(args: &CommandPrepareISO) -> Result<()> {
+ match Path::try_exists(&args.source) {
+ Ok(true) => (),
+ Ok(false) => bail!("Source file does not exist."),
+ Err(_) => bail!("Source file does not exist."),
+ }
+
+ match Command::new("xorriso")
+ .arg("-dev")
+ .arg(&args.source)
+ .arg("-find")
+ .arg(PROXMOX_ISO_FLAG)
+ .stderr(Stdio::null())
+ .stdout(Stdio::null())
+ .status()
+ {
+ Ok(v) => {
+ if !v.success() {
+ bail!("The source ISO file is not able to be installed automatically. Please try a more current one.");
+ }
+ }
+ Err(_) => bail!("Could not run 'xorriso'. Please install it."),
+ };
+
+ Ok(())
+}
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand
2024-04-18 8:48 ` Christoph Heiss
2024-04-18 9:13 ` Aaron Lauterer
@ 2024-04-18 13:39 ` Thomas Lamprecht
1 sibling, 0 replies; 41+ messages in thread
From: Thomas Lamprecht @ 2024-04-18 13:39 UTC (permalink / raw)
To: Proxmox VE development discussion, Christoph Heiss, Aaron Lauterer
Am 18/04/2024 um 10:48 schrieb Christoph Heiss:
> Do we really need _yet another_ crate dependency for that? Below is a
> check / bail! anyway when running the command proper.
>
> And if we really want a explicit check beforehand, I'd just do something
> like
>
> fn which(name: &str) -> Result<()> {
> match Command::new(name).output() {
> Ok(_) => Ok(()),
> Err(err) => Err(err.into()),
> }
> }
yeah, +1. I'd just catch the error and execute and wrap it as nicely as
possibly to convey to the user that xorriso is required somewhere in $PATH.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2024-04-18 13:39 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-17 12:30 [pve-devel] [PATCH installer v6 00/36] add automated/unattended installation Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 02/36] common: make InstallZfsOption members public Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 03/36] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 04/36] common: utils: add deserializer for CidrAddress Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 05/36] common: options: add Deserialize trait Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 06/36] low-level: add dump-udev command Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 07/36] add auto-installer crate Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 08/36] auto-installer: add dependencies Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 09/36] auto-installer: add answer file definition Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 10/36] auto-installer: add struct to hold udev info Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 11/36] auto-installer: add utils Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 12/36] auto-installer: add simple logging Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 13/36] auto-installer: add tests for answer file parsing Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 14/36] auto-installer: add auto-installer binary Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 15/36] auto-installer: add fetch answer binary Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 16/36] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 17/36] auto-installer: use glob crate for pattern matching Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 18/36] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 19/36] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 20/36] common: add Display trait to ProxmoxProduct Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 21/36] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 22/36] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 23/36] auto-installer: fetch: add http post utility module Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 24/36] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 25/36] control: update build depends for auto installer Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 26/36] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
2024-04-17 12:30 ` [pve-devel] [PATCH installer v6 27/36] low-level: write low level config to /tmp Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 28/36] common: add deserializer for FsType Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 29/36] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 30/36] add proxmox-chroot utility Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 31/36] auto-installer: answer: deny unknown fields Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 32/36] fetch-answer: move get_answer_file to utils Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 33/36] auto-installer: utils: define ISO specified settings Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 34/36] fetch-answer: use ISO specified configurations Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 35/36] fetch-answer: dpcp: improve logging of steps taken Aaron Lauterer
2024-04-17 12:31 ` [pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand Aaron Lauterer
2024-04-18 8:48 ` Christoph Heiss
2024-04-18 9:13 ` Aaron Lauterer
2024-04-18 13:39 ` Thomas Lamprecht
2024-04-18 11:38 ` [pve-devel] [PATCH installer v6 36/36 follow-up] " Aaron Lauterer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox