* [pve-devel] [PATCH v2 00/22] add automated/unattended installation
@ 2024-02-21 11:07 Aaron Lauterer
2024-02-21 11:07 ` [pve-devel] [PATCH v2 01/22] tui: common: move InstallConfig struct to common crate Aaron Lauterer
` (23 more replies)
0 siblings, 24 replies; 29+ messages in thread
From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw)
To: pve-devel
This patch series adds the possibility to do an automated / unattended
installation of Proxmox VE.
It assumes that the patch series to use JSON output on the
low-level-installer is already applied [1].
The overall idea is that we will have a dedicated ISO for the unattended
installation. It should be configured in such a way that it will start
the installation without any user interaction.
Though the integration in the installation environmend isn't part of
this patch series.
The information for the installer that is usually gathered interactively
from the user is provided via an `answer.toml` file.
The answer file allows to select disks and the network card via filters.
The installer also allows to run custom commands pre and post
installation. This should give users plenty of possibilities to either
further customize/prepare the installation or integrate it into a larger
automated installation setup.
For example, one could issue HTTP requests to signal the status and
progress of the installation.
The install environment needs to call the 'proxmox-fetch-answer' binary.
It tries to find the answer file and once found, will start the
'proxmox-auto-installer' binary and pass the contents to it via stdin.
The auto-installer then parses the answer file and determines what
parameters need to be passed to the low-level installer. For example,
which disks and NIC to use, network IP settings and so forth.
The current status reporting of the actual installation is kept rather
simple.
Both binaries log into the tmp directory.
There is a third binary, the 'proxmox-autoinst-helper'. It provides a
few subcommands, from the help:
answer Validate if an answer file is formatted correctly
device-match Test which devices the given filter matches against
device-info Show device information that can be used for filters
identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file
The fetch-answer binary is trying to get an answer file. It does so by
first searching for a partition/FS labeled `proxmoxinst`, or all upper
case, and an `answer.toml` in there. This could be provided by another
USB flash drive.
If that is not successful, the next step is to send an HTTP POST request
to a URL to get the TOML contents in return. A POST request was chosen
because we also send information to identify the host in JSON format.
The question then is, where to get that URL from. Right now, there are
two options implemented. The first is looking for a custom DHCP option
and the second is querying for a TXT record in the `proxmoxinst`
subdomain of the search domain.
It is possible to provide a SHA256 fingerprint of the SSL cert used by
the answer server. The safest option is to place a
`cert_fingerprint.txt` file in the same `proxmoxinst` partition as where
you alternatively would place the `answer.toml`.
If that is not found, then it can be provided by a second custom DHCP
option or placed as TXT record in the subdomain `proxmoxinst-fp`.
Other plans / ideas for the future:
* add a chroot helper tool to make it easier to modify the installation
in a post-command (I might send a follow up patch in the next days)
* add option to define remote SSH access (password and,or public key).
This could make remote debugging in case of problems easier
Regarding the patch series itself:
01-03 are needed to move some code into the common crate and
make structs/functions already in the common crate accessible.
I did split up the individual parts of the auto installer into their own
patches as much as possible, and (hopefully) in the order they depend on
each other.
Patches after the `unconfigured` one (14), switch the pattern matching
to the glob crate, add the helper tool and the fetching via HTTP.
The last patch updates the build dependencies. I am not sure though if
I did it correctly so take it with a grain of salt.
I did skip on the doc patch this time as it needs more time and will be
either sent as a dedicated patch or follow up to this series.
Areas that can be improved/extended:
* Testing possibility integrated in the Makefile
* Documentation: explain process, additional examples for answer.toml
Changes since v1:
* incorporated feedback (thx @cheiss)
* the proxmox-installer-filter tool got renamed and extended to the
proxmox-autoinst-helper
* added HTTP fetch functionality
* moved the fetch_plugins/utils.rs into its own subdirectory as it grew
quite a bit
* added builddeps
[0] https://lists.proxmox.com/pipermail/pve-devel/2023-September/059020.html
[1] https://lists.proxmox.com/pipermail/pve-devel/2023-December/060961.html
Aaron Lauterer (22):
tui: common: move InstallConfig struct to common crate
common: make InstallZfsOption members public
common: tui: use BTreeMap for predictable ordering
low-level: add dump-udev command
add auto-installer crate
auto-installer: add dependencies
auto-installer: add answer file definition
auto-installer: add struct to hold udev info
auto-installer: add utils
auto-installer: add simple logging
auto-installer: add tests for answer file parsing
auto-installer: add auto-installer binary
auto-installer: add fetch answer binary
unconfigured: add proxauto as option to start auto installer
auto-installer: use glob crate for pattern matching
auto-installer: utils: make get_udev_index functions public
auto-installer: add proxmox-autoinst-helper tool
auto-installer: fetch: add gathering of system identifiers and
restructure code
auto-installer: helper: add subcommand to view indentifiers
auto-installer: fetch: add http post utility module
auto-installer: fetch: add http plugin to fetch answer
control: update build depends for auto installer
Cargo.toml | 1 +
Makefile | 7 +-
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 +++
debian/control | 10 +
proxmox-auto-installer/Cargo.toml | 25 +
proxmox-auto-installer/src/answer.rs | 133 +++++
.../src/bin/proxmox-auto-installer.rs | 193 ++++++++
.../src/bin/proxmox-autoinst-helper.rs | 333 +++++++++++++
.../src/bin/proxmox-fetch-answer.rs | 76 +++
.../src/fetch_plugins/http.rs | 185 +++++++
.../src/fetch_plugins/mod.rs | 3 +
.../src/fetch_plugins/partition.rs | 32 ++
.../src/fetch_plugins/utils/mod.rs | 114 +++++
.../src/fetch_plugins/utils/post.rs | 93 ++++
.../src/fetch_plugins/utils/sysinfo.rs | 200 ++++++++
proxmox-auto-installer/src/lib.rs | 5 +
proxmox-auto-installer/src/log.rs | 38 ++
proxmox-auto-installer/src/udevinfo.rs | 9 +
proxmox-auto-installer/src/utils.rs | 459 ++++++++++++++++++
proxmox-auto-installer/tests/parse-answer.rs | 102 ++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 ++
.../resources/parse_answer/disk_match.toml | 14 +
.../parse_answer/disk_match_all.json | 26 +
.../parse_answer/disk_match_all.toml | 16 +
.../parse_answer/disk_match_any.json | 33 ++
.../parse_answer/disk_match_any.toml | 16 +
.../tests/resources/parse_answer/minimal.json | 17 +
.../tests/resources/parse_answer/minimal.toml | 14 +
.../resources/parse_answer/nic_matching.json | 17 +
.../resources/parse_answer/nic_matching.toml | 19 +
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +
.../resources/parse_answer/specific_nic.toml | 19 +
.../tests/resources/parse_answer/zfs.json | 27 ++
.../tests/resources/parse_answer/zfs.toml | 19 +
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
proxmox-installer-common/src/setup.rs | 100 +++-
proxmox-low-level-installer | 13 +
proxmox-tui-installer/src/options.rs | 4 +-
proxmox-tui-installer/src/setup.rs | 100 +---
.../src/views/install_progress.rs | 4 +-
unconfigured.sh | 17 +
46 files changed, 2493 insertions(+), 109 deletions(-)
create mode 100644 Proxmox/Sys/Udev.pm
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/http.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/post.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
create mode 100644 proxmox-auto-installer/src/lib.rs
create mode 100644 proxmox-auto-installer/src/log.rs
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
create mode 100644 proxmox-auto-installer/src/utils.rs
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
--
2.39.2
^ permalink raw reply [flat|nested] 29+ messages in thread* [pve-devel] [PATCH v2 01/22] tui: common: move InstallConfig struct to common crate 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 02/22] common: make InstallZfsOption members public Aaron Lauterer ` (22 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel It describes the data structure expected by the low-level-installer. We do this so we can use it in more than the TUI installer, for example the planned auto installer. Make the members public so we can easily implement a custom From method for each dependent crate. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-installer-common/src/setup.rs | 86 +++++++++++++++- proxmox-tui-installer/src/setup.rs | 98 +------------------ .../src/views/install_progress.rs | 4 +- 3 files changed, 90 insertions(+), 98 deletions(-) diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs index 472e1f2..03beb77 100644 --- a/proxmox-installer-common/src/setup.rs +++ b/proxmox-installer-common/src/setup.rs @@ -12,7 +12,10 @@ use std::{ use serde::{de, Deserialize, Deserializer, Serialize, Serializer}; use crate::{ - options::{Disk, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption}, + options::{ + BtrfsRaidLevel, Disk, FsType, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption, + ZfsRaidLevel, + }, utils::CidrAddress, }; @@ -387,3 +390,84 @@ pub fn spawn_low_level_installer(test_mode: bool) -> io::Result<process::Child> .stdout(Stdio::piped()) .spawn() } + +/// See Proxmox::Install::Config +#[derive(Serialize)] +pub struct InstallConfig { + pub autoreboot: usize, + + #[serde(serialize_with = "serialize_fstype")] + pub filesys: FsType, + pub hdsize: f64, + #[serde(skip_serializing_if = "Option::is_none")] + pub swapsize: Option<f64>, + #[serde(skip_serializing_if = "Option::is_none")] + pub maxroot: Option<f64>, + #[serde(skip_serializing_if = "Option::is_none")] + pub minfree: Option<f64>, + #[serde(skip_serializing_if = "Option::is_none")] + pub maxvz: Option<f64>, + + #[serde(skip_serializing_if = "Option::is_none")] + pub zfs_opts: Option<InstallZfsOption>, + + #[serde( + serialize_with = "serialize_disk_opt", + skip_serializing_if = "Option::is_none" + )] + pub target_hd: Option<Disk>, + #[serde(skip_serializing_if = "HashMap::is_empty")] + pub disk_selection: HashMap<String, String>, + + pub country: String, + pub timezone: String, + pub keymap: String, + + pub password: String, + pub mailto: String, + + pub mngmt_nic: String, + + pub hostname: String, + pub domain: String, + #[serde(serialize_with = "serialize_as_display")] + pub cidr: CidrAddress, + pub gateway: IpAddr, + pub dns: IpAddr, +} + +fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error> +where + S: Serializer, +{ + if let Some(disk) = value { + serializer.serialize_str(&disk.path) + } else { + serializer.serialize_none() + } +} + +fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error> +where + S: Serializer, +{ + use FsType::*; + let value = match value { + // proxinstall::$fssetup + Ext4 => "ext4", + Xfs => "xfs", + // proxinstall::get_zfs_raid_setup() + Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)", + Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)", + Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)", + Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)", + Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)", + Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)", + // proxinstall::get_btrfs_raid_setup() + Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)", + Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)", + Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)", + }; + + serializer.collect_str(value) +} diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs index 79421d7..e816c12 100644 --- a/proxmox-tui-installer/src/setup.rs +++ b/proxmox-tui-installer/src/setup.rs @@ -1,59 +1,11 @@ -use std::{collections::HashMap, fmt, net::IpAddr}; - -use serde::{Serialize, Serializer}; +use std::collections::HashMap; use crate::options::InstallerOptions; use proxmox_installer_common::{ - options::{AdvancedBootdiskOptions, BtrfsRaidLevel, Disk, FsType, ZfsRaidLevel}, - setup::InstallZfsOption, - utils::CidrAddress, + options::AdvancedBootdiskOptions, + setup::InstallConfig, }; -/// See Proxmox::Install::Config -#[derive(Serialize)] -pub struct InstallConfig { - autoreboot: usize, - - #[serde(serialize_with = "serialize_fstype")] - filesys: FsType, - hdsize: f64, - #[serde(skip_serializing_if = "Option::is_none")] - swapsize: Option<f64>, - #[serde(skip_serializing_if = "Option::is_none")] - maxroot: Option<f64>, - #[serde(skip_serializing_if = "Option::is_none")] - minfree: Option<f64>, - #[serde(skip_serializing_if = "Option::is_none")] - maxvz: Option<f64>, - - #[serde(skip_serializing_if = "Option::is_none")] - zfs_opts: Option<InstallZfsOption>, - - #[serde( - serialize_with = "serialize_disk_opt", - skip_serializing_if = "Option::is_none" - )] - target_hd: Option<Disk>, - #[serde(skip_serializing_if = "HashMap::is_empty")] - disk_selection: HashMap<String, String>, - - country: String, - timezone: String, - keymap: String, - - password: String, - mailto: String, - - mngmt_nic: String, - - hostname: String, - domain: String, - #[serde(serialize_with = "serialize_as_display")] - cidr: CidrAddress, - gateway: IpAddr, - dns: IpAddr, -} - impl From<InstallerOptions> for InstallConfig { fn from(options: InstallerOptions) -> Self { let mut config = Self { @@ -121,47 +73,3 @@ impl From<InstallerOptions> for InstallConfig { config } } - -fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error> -where - S: Serializer, -{ - if let Some(disk) = value { - serializer.serialize_str(&disk.path) - } else { - serializer.serialize_none() - } -} - -fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error> -where - S: Serializer, -{ - use FsType::*; - let value = match value { - // proxinstall::$fssetup - Ext4 => "ext4", - Xfs => "xfs", - // proxinstall::get_zfs_raid_setup() - Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)", - Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)", - Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)", - Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)", - Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)", - Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)", - // proxinstall::get_btrfs_raid_setup() - Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)", - Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)", - Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)", - }; - - serializer.collect_str(value) -} - -fn serialize_as_display<S, T>(value: &T, serializer: S) -> Result<S::Ok, S::Error> -where - S: Serializer, - T: fmt::Display, -{ - serializer.collect_str(value) -} diff --git a/proxmox-tui-installer/src/views/install_progress.rs b/proxmox-tui-installer/src/views/install_progress.rs index 4626fe4..409c31a 100644 --- a/proxmox-tui-installer/src/views/install_progress.rs +++ b/proxmox-tui-installer/src/views/install_progress.rs @@ -12,8 +12,8 @@ use std::{ time::Duration, }; -use crate::{abort_install_button, prompt_dialog, setup::InstallConfig, InstallerState}; -use proxmox_installer_common::setup::spawn_low_level_installer; +use crate::{abort_install_button, prompt_dialog, InstallerState}; +use proxmox_installer_common::setup::{InstallConfig, spawn_low_level_installer}; pub struct InstallProgressView { view: PaddedView<LinearLayout>, -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 02/22] common: make InstallZfsOption members public 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 01/22] tui: common: move InstallConfig struct to common crate Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 03/22] common: tui: use BTreeMap for predictable ordering Aaron Lauterer ` (21 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel as they will be used directly by the auto installer Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-installer-common/src/setup.rs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs index 03beb77..1bc4cb7 100644 --- a/proxmox-installer-common/src/setup.rs +++ b/proxmox-installer-common/src/setup.rs @@ -147,13 +147,13 @@ pub fn installer_setup(in_test_mode: bool) -> Result<(SetupInfo, LocaleInfo, Run #[derive(Serialize)] pub struct InstallZfsOption { - ashift: usize, + pub ashift: usize, #[serde(serialize_with = "serialize_as_display")] - compress: ZfsCompressOption, + pub compress: ZfsCompressOption, #[serde(serialize_with = "serialize_as_display")] - checksum: ZfsChecksumOption, - copies: usize, - arc_max: usize, + pub checksum: ZfsChecksumOption, + pub copies: usize, + pub arc_max: usize, } impl From<ZfsBootdiskOptions> for InstallZfsOption { -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 03/22] common: tui: use BTreeMap for predictable ordering 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 01/22] tui: common: move InstallConfig struct to common crate Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 02/22] common: make InstallZfsOption members public Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 04/22] low-level: add dump-udev command Aaron Lauterer ` (20 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel necessary for the disk selection and network interfaces maps to have tests with results that can be compared without much additional effort. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-installer-common/src/setup.rs | 8 ++++---- proxmox-tui-installer/src/options.rs | 4 ++-- proxmox-tui-installer/src/setup.rs | 4 ++-- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs index 1bc4cb7..8432a2c 100644 --- a/proxmox-installer-common/src/setup.rs +++ b/proxmox-installer-common/src/setup.rs @@ -1,6 +1,6 @@ use std::{ cmp, - collections::HashMap, + collections::{BTreeMap, HashMap}, fmt, fs::File, io::{self, BufReader}, @@ -297,7 +297,7 @@ pub struct NetworkInfo { /// Maps devices to their configuration, if it has a usable configuration. /// (Contains no entries for devices with only link-local addresses.) #[serde(default)] - pub interfaces: HashMap<String, Interface>, + pub interfaces: BTreeMap<String, Interface>, /// The hostname of this machine, if set by the DHCP server. pub hostname: Option<String>, @@ -416,8 +416,8 @@ pub struct InstallConfig { skip_serializing_if = "Option::is_none" )] pub target_hd: Option<Disk>, - #[serde(skip_serializing_if = "HashMap::is_empty")] - pub disk_selection: HashMap<String, String>, + #[serde(skip_serializing_if = "BTreeMap::is_empty")] + pub disk_selection: BTreeMap<String, String>, pub country: String, pub timezone: String, diff --git a/proxmox-tui-installer/src/options.rs b/proxmox-tui-installer/src/options.rs index 094a430..73fbf2a 100644 --- a/proxmox-tui-installer/src/options.rs +++ b/proxmox-tui-installer/src/options.rs @@ -76,7 +76,7 @@ mod tests { utils::{CidrAddress, Fqdn}, }; use std::net::{IpAddr, Ipv4Addr}; - use std::{collections::HashMap, path::PathBuf}; + use std::{collections::BTreeMap, path::PathBuf}; fn dummy_setup_info() -> SetupInfo { SetupInfo { @@ -99,7 +99,7 @@ mod tests { fn network_options_from_setup_network_info() { let setup = dummy_setup_info(); - let mut interfaces = HashMap::new(); + let mut interfaces = BTreeMap::new(); interfaces.insert( "eth0".to_owned(), Interface { diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs index e816c12..248f86e 100644 --- a/proxmox-tui-installer/src/setup.rs +++ b/proxmox-tui-installer/src/setup.rs @@ -1,4 +1,4 @@ -use std::collections::HashMap; +use std::collections::BTreeMap; use crate::options::InstallerOptions; use proxmox_installer_common::{ @@ -19,7 +19,7 @@ impl From<InstallerOptions> for InstallConfig { maxvz: None, zfs_opts: None, target_hd: None, - disk_selection: HashMap::new(), + disk_selection: BTreeMap::new(), country: options.timezone.country, timezone: options.timezone.timezone, -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 04/22] low-level: add dump-udev command 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (2 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 03/22] common: tui: use BTreeMap for predictable ordering Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 05/22] add auto-installer crate Aaron Lauterer ` (19 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Fetches UDEV device properties prepended with 'E:' for NICs and disks. The result is stored in its own JSON file. This information is needed to filter for specific devices. Mainly for the auto-installer for now. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- Proxmox/Makefile | 1 + Proxmox/Sys/Udev.pm | 54 +++++++++++++++++++++++++++++++++++++ proxmox-low-level-installer | 13 +++++++++ 3 files changed, 68 insertions(+) create mode 100644 Proxmox/Sys/Udev.pm diff --git a/Proxmox/Makefile b/Proxmox/Makefile index d49da80..9561d9b 100644 --- a/Proxmox/Makefile +++ b/Proxmox/Makefile @@ -16,6 +16,7 @@ PERL_MODULES=\ Sys/Command.pm \ Sys/File.pm \ Sys/Net.pm \ + Sys/Udev.pm \ UI.pm \ UI/Base.pm \ UI/Gtk3.pm \ diff --git a/Proxmox/Sys/Udev.pm b/Proxmox/Sys/Udev.pm new file mode 100644 index 0000000..69d674f --- /dev/null +++ b/Proxmox/Sys/Udev.pm @@ -0,0 +1,54 @@ +package Proxmox::Sys::Udev; + +use strict; +use warnings; + +use base qw(Exporter); +our @EXPORT_OK = qw(disk_details); + +my $udev_regex = '^E: ([A-Z_]*)=(.*)$'; + +my sub fetch_udevadm_info { + my ($path) = @_; + + my $info = `udevadm info --path $path --query all`; + if (!$info) { + warn "no details found for device '${path}'\n"; + next; + } + my $details = {}; + for my $line (split('\n', $info)) { + if ($line =~ m/$udev_regex/) { + $details->{$1} = $2; + } + } + return $details; +} + +# return hash of E: properties returned by udevadm +sub disk_details { + my $result = {}; + for my $data (@{Proxmox::Sys::Block::get_cached_disks()}) { + my $index = @$data[0]; + my $bd = @$data[5]; + $result->{$index} = fetch_udevadm_info($bd); + } + return $result; +} + + +sub nic_details { + my $nic_path = "/sys/class/net"; + my $result = {}; + + my $nics = Proxmox::Sys::Net::get_ip_config()->{ifaces}; + + for my $index (keys %$nics) { + my $name = $nics->{$index}->{name}; + my $nic = "${nic_path}/${name}"; + $result->{$name} = fetch_udevadm_info($nic); + } + return $result; +} + +1; diff --git a/proxmox-low-level-installer b/proxmox-low-level-installer index d127a40..531251d 100755 --- a/proxmox-low-level-installer +++ b/proxmox-low-level-installer @@ -21,6 +21,7 @@ use Time::HiRes qw(usleep); use Proxmox::Install::ISOEnv; use Proxmox::Install::RunEnv; +use Proxmox::Sys::Udev; use Proxmox::Sys::File qw(file_write_all); @@ -31,6 +32,7 @@ use Proxmox::UI; my $commands = { 'dump-env' => 'Dump the current ISO and Hardware environment to base the installer UI on.', + 'dump-udev' => 'Dump disk and network device info. Used for the auto installation.', 'start-session' => 'Start an installation session, with command and result transmitted via stdin/out', 'start-session-test' => 'Start an installation TEST session, with command and result transmitted via stdin/out', 'help' => 'Output this usage help.', @@ -114,6 +116,17 @@ if ($cmd eq 'dump-env') { my $run_env = Proxmox::Install::RunEnv::query_installation_environment(); my $run_env_serialized = to_json($run_env, {canonical => 1, utf8 => 1}) ."\n"; file_write_all($run_env_file, $run_env_serialized); +} elsif ($cmd eq 'dump-udev') { + my $out_dir = $env->{locations}->{run}; + make_path($out_dir); + die "failed to create output directory '$out_dir'\n" if !-d $out_dir; + + my $output = {}; + $output->{disks} = Proxmox::Sys::Udev::disk_details(); + $output->{nics} = Proxmox::Sys::Udev::nic_details(); + + my $output_serialized = to_json($output, {canonical => 1, utf8 => 1}) ."\n"; + file_write_all("$out_dir/run-env-udev.json", $output_serialized); } elsif ($cmd eq 'start-session') { Proxmox::UI::init_stdio({}, $env); read_and_merge_config(); -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 05/22] add auto-installer crate 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (3 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 04/22] low-level: add dump-udev command Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 06/22] auto-installer: add dependencies Aaron Lauterer ` (18 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- Cargo.toml | 1 + Makefile | 1 + proxmox-auto-installer/Cargo.toml | 10 ++++++++++ proxmox-auto-installer/src/lib.rs | 0 4 files changed, 12 insertions(+) create mode 100644 proxmox-auto-installer/Cargo.toml create mode 100644 proxmox-auto-installer/src/lib.rs diff --git a/Cargo.toml b/Cargo.toml index c1bd578..7017ac5 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,5 +1,6 @@ [workspace] members = [ + "proxmox-auto-installer", "proxmox-installer-common", "proxmox-tui-installer", ] diff --git a/Makefile b/Makefile index f0c361b..57bd7ae 100644 --- a/Makefile +++ b/Makefile @@ -42,6 +42,7 @@ $(BUILDDIR): policy-disable-rc.d \ proxinstall \ proxmox-low-level-installer \ + proxmox-auto-installer/ \ proxmox-tui-installer/ \ proxmox-installer-common/ \ spice-vdagent.sh \ diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml new file mode 100644 index 0000000..75cfb2c --- /dev/null +++ b/proxmox-auto-installer/Cargo.toml @@ -0,0 +1,10 @@ +[package] +name = "proxmox-auto-installer" +version = "0.1.0" +edition = "2021" +authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ] +license = "AGPL-3" +exclude = [ "build", "debian" ] +homepage = "https://www.proxmox.com" + +[dependencies] diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs new file mode 100644 index 0000000..e69de29 -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 06/22] auto-installer: add dependencies 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (4 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 05/22] add auto-installer crate Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 07/22] auto-installer: add answer file definition Aaron Lauterer ` (17 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- changes since v1: * toml switched from 0.5 to 0.7 proxmox-auto-installer/Cargo.toml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 75cfb2c..67218dd 100644 --- a/proxmox-auto-installer/Cargo.toml +++ b/proxmox-auto-installer/Cargo.toml @@ -8,3 +8,7 @@ exclude = [ "build", "debian" ] homepage = "https://www.proxmox.com" [dependencies] +proxmox-installer-common = { path = "../proxmox-installer-common" } +serde = { version = "1.0", features = ["derive"] } +serde_json = "1.0" +toml = "0.7" -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 07/22] auto-installer: add answer file definition 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (5 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 06/22] auto-installer: add dependencies Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 08/22] auto-installer: add struct to hold udev info Aaron Lauterer ` (16 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- changes since v1: * added default traits and switched to them proxmox-auto-installer/src/answer.rs | 132 +++++++++++++++++++++++++++ proxmox-auto-installer/src/lib.rs | 1 + 2 files changed, 133 insertions(+) create mode 100644 proxmox-auto-installer/src/answer.rs diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs new file mode 100644 index 0000000..8ba677a --- /dev/null +++ b/proxmox-auto-installer/src/answer.rs @@ -0,0 +1,132 @@ +use serde::{Deserialize, Serialize}; +use std::collections::BTreeMap; + +#[derive(Clone, Deserialize, Debug)] +#[serde(rename_all = "lowercase")] +pub struct Answer { + pub global: Global, + pub network: Network, + pub disks: Disks, +} + +#[derive(Clone, Deserialize, Debug)] +pub struct Global { + pub country: String, + pub fqdn: String, + pub keyboard: String, + pub mailto: String, + pub timezone: String, + pub password: String, + pub pre_command: Option<Vec<String>>, + pub post_command: Option<Vec<String>>, + pub reboot_on_error: Option<bool>, +} + +#[derive(Clone, Deserialize, Debug)] +pub struct Network { + pub use_dhcp: Option<bool>, + pub cidr: Option<String>, + pub dns: Option<String>, + pub gateway: Option<String>, + // use BTreeMap to have keys sorted + pub filter: Option<BTreeMap<String, String>>, +} + +#[derive(Clone, Deserialize, Debug)] +pub struct Disks { + pub filesystem: Option<Filesystem>, + pub disk_selection: Option<Vec<String>>, + pub filter_match: Option<FilterMatch>, + // use BTreeMap to have keys sorted + pub filter: Option<BTreeMap<String, String>>, + pub zfs: Option<ZfsOptions>, + pub lvm: Option<LvmOptions>, + pub btrfs: Option<BtrfsOptions>, +} + +#[derive(Clone, Deserialize, Debug, PartialEq)] +#[serde(rename_all = "lowercase")] +pub enum FilterMatch { + Any, + All, +} + +#[derive(Clone, Deserialize, Serialize, Debug)] +#[serde(rename_all = "kebab-case")] +pub enum Filesystem { + Ext4, + Xfs, + ZfsRaid0, + ZfsRaid1, + ZfsRaid10, + ZfsRaidZ1, + ZfsRaidZ2, + ZfsRaidZ3, + BtrfsRaid0, + BtrfsRaid1, + BtrfsRaid10, +} + +#[derive(Clone, Default, Deserialize, Debug)] +pub struct ZfsOptions { + pub ashift: Option<usize>, + pub arc_max: Option<usize>, + pub checksum: Option<ZfsChecksumOption>, + pub compress: Option<ZfsCompressOption>, + pub copies: Option<usize>, + pub hdsize: Option<f64>, +} + +impl ZfsOptions { + pub fn new() -> ZfsOptions { + ZfsOptions::default() + } +} + +#[derive(Copy, Clone, Debug, Default, Eq, PartialEq, Deserialize)] +#[serde(rename_all(deserialize = "lowercase"))] +pub enum ZfsCompressOption { + #[default] + On, + Off, + Lzjb, + Lz4, + Zle, + Gzip, + Zstd, +} + +#[derive(Copy, Clone, Debug, Default, Eq, PartialEq, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum ZfsChecksumOption { + #[default] + On, + Fletcher4, + Sha256, +} + +#[derive(Clone, Default, Deserialize, Serialize, Debug)] +pub struct LvmOptions { + pub hdsize: Option<f64>, + pub swapsize: Option<f64>, + pub maxroot: Option<f64>, + pub maxvz: Option<f64>, + pub minfree: Option<f64>, +} + +impl LvmOptions { + pub fn new() -> LvmOptions { + LvmOptions::default() + } +} + +#[derive(Clone, Default, Deserialize, Serialize, Debug)] +pub struct BtrfsOptions { + pub hdsize: Option<f64>, +} + +impl BtrfsOptions { + pub fn new() -> BtrfsOptions { + BtrfsOptions::default() + } +} diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs index e69de29..7813b98 100644 --- a/proxmox-auto-installer/src/lib.rs +++ b/proxmox-auto-installer/src/lib.rs @@ -0,0 +1 @@ +pub mod answer; -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 08/22] auto-installer: add struct to hold udev info 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (6 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 07/22] auto-installer: add answer file definition Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 09/22] auto-installer: add utils Aaron Lauterer ` (15 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/src/lib.rs | 1 + proxmox-auto-installer/src/udevinfo.rs | 9 +++++++++ 2 files changed, 10 insertions(+) create mode 100644 proxmox-auto-installer/src/udevinfo.rs diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs index 7813b98..8cda416 100644 --- a/proxmox-auto-installer/src/lib.rs +++ b/proxmox-auto-installer/src/lib.rs @@ -1 +1,2 @@ pub mod answer; +pub mod udevinfo; diff --git a/proxmox-auto-installer/src/udevinfo.rs b/proxmox-auto-installer/src/udevinfo.rs new file mode 100644 index 0000000..a6b61b5 --- /dev/null +++ b/proxmox-auto-installer/src/udevinfo.rs @@ -0,0 +1,9 @@ +use serde::Deserialize; +use std::collections::BTreeMap; + +#[derive(Clone, Deserialize, Debug)] +pub struct UdevInfo { + // use BTreeMap to have keys sorted + pub disks: BTreeMap<String, BTreeMap<String, String>>, + pub nics: BTreeMap<String, BTreeMap<String, String>>, +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 09/22] auto-installer: add utils 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (7 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 08/22] auto-installer: add struct to hold udev info Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 10/22] auto-installer: add simple logging Aaron Lauterer ` (14 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel contains several utility structs and functions. For example: a simple pattern matcher that matches wildcards at the beginning or end of the filter. It currently uses a dedicated function (parse_answer) to generate the InstallConfig struct instead of a From implementation. This is because for now the source data is spread over several other structs in comparison to one in the TUI installer. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/src/lib.rs | 1 + proxmox-auto-installer/src/utils.rs | 471 ++++++++++++++++++++++++++++ 2 files changed, 472 insertions(+) create mode 100644 proxmox-auto-installer/src/utils.rs diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs index 8cda416..72884c1 100644 --- a/proxmox-auto-installer/src/lib.rs +++ b/proxmox-auto-installer/src/lib.rs @@ -1,2 +1,3 @@ pub mod answer; pub mod udevinfo; +pub mod utils; diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs new file mode 100644 index 0000000..23bedd9 --- /dev/null +++ b/proxmox-auto-installer/src/utils.rs @@ -0,0 +1,471 @@ +use anyhow::{anyhow, bail, Context, Result}; +use log::info; +use std::{ + collections::BTreeMap, + net::IpAddr, + process::{Command, Stdio}, + str::FromStr, +}; + +use crate::{ + answer, + answer::{Answer, FilterMatch}, + udevinfo::UdevInfo, +}; +use proxmox_installer_common::{ + options::{ + BtrfsRaidLevel, FsType, NetworkOptions, ZfsChecksumOption, ZfsCompressOption, ZfsRaidLevel, + }, + setup::{InstallConfig, InstallZfsOption, LocaleInfo, RuntimeInfo, SetupInfo}, + utils::{CidrAddress, Fqdn}, +}; +use serde::Deserialize; + +/// Supports the globbing character '*' at the beginning, end or both of the pattern. +/// Globbing within the pattern is not supported +fn find_with_glob(pattern: &str, value: &str) -> bool { + let globbing_symbol = '*'; + let mut start_glob = false; + let mut end_glob = false; + let mut pattern = pattern; + + if pattern.starts_with(globbing_symbol) { + start_glob = true; + pattern = &pattern[1..]; + } + + if pattern.ends_with(globbing_symbol) { + end_glob = true; + pattern = &pattern[..pattern.len() - 1] + } + + match (start_glob, end_glob) { + (true, true) => value.contains(pattern), + (true, false) => value.ends_with(pattern), + (false, true) => value.starts_with(pattern), + _ => value == pattern, + } +} + +pub fn get_network_settings( + answer: &Answer, + udev_info: &UdevInfo, + runtime_info: &RuntimeInfo, + setup_info: &SetupInfo, +) -> Result<NetworkOptions> { + let mut network_options = NetworkOptions::defaults_from(setup_info, &runtime_info.network); + + info!("Setting network configuration"); + + // Always use the FQDN from the answer file + network_options.fqdn = Fqdn::from(answer.global.fqdn.as_str()) + .map_err(|err| anyhow!("Error parsing FQDN: {err}"))?; + + if answer.network.use_dhcp.is_none() || !answer.network.use_dhcp.unwrap() { + network_options.address = CidrAddress::from_str( + answer + .network + .cidr + .clone() + .context("No CIDR defined")? + .as_str(), + ) + .map_err(|_| anyhow!("Error parsing CIDR"))?; + network_options.dns_server = IpAddr::from_str( + answer + .network + .dns + .clone() + .context("No DNS server defined")? + .as_str(), + ) + .map_err(|_| anyhow!("Error parsing DNS server"))?; + network_options.gateway = IpAddr::from_str( + answer + .network + .gateway + .clone() + .expect("No gateway defined") + .as_str(), + ) + .map_err(|_| anyhow!("Error parsing gateway"))?; + network_options.ifname = + get_single_udev_index(answer.network.filter.clone().unwrap(), &udev_info.nics)? + } + + Ok(network_options) +} + +fn get_single_udev_index( + filter: BTreeMap<String, String>, + udev_list: &BTreeMap<String, BTreeMap<String, String>>, +) -> Result<String> { + if filter.is_empty() { + bail!("no filter defined"); + } + let mut dev_index: Option<String> = None; + 'outer: for (dev, dev_values) in udev_list { + for (filter_key, filter_value) in &filter { + for (udev_key, udev_value) in dev_values { + if udev_key == filter_key && find_with_glob(filter_value, udev_value) { + dev_index = Some(dev.clone()); + break 'outer; // take first match + } + } + } + } + if dev_index.is_none() { + bail!("filter did not match any device"); + } + + Ok(dev_index.unwrap()) +} + +fn get_matched_udev_indexes( + filter: BTreeMap<String, String>, + udev_list: &BTreeMap<String, BTreeMap<String, String>>, + match_all: bool, +) -> Result<Vec<String>> { + let mut matches = vec![]; + for (dev, dev_values) in udev_list { + let mut did_match_once = false; + let mut did_match_all = true; + for (filter_key, filter_value) in &filter { + for (udev_key, udev_value) in dev_values { + if udev_key == filter_key && find_with_glob(filter_value, udev_value) { + did_match_once = true; + } else if udev_key == filter_key { + did_match_all = false; + } + } + } + if (match_all && did_match_all) || (!match_all && did_match_once) { + matches.push(dev.clone()); + } + } + if matches.is_empty() { + bail!("filter did not match any devices"); + } + matches.sort(); + Ok(matches) +} + +pub fn set_disks( + answer: &Answer, + udev_info: &UdevInfo, + runtime_info: &RuntimeInfo, + config: &mut InstallConfig, +) -> Result<()> { + match config.filesys { + FsType::Ext4 | FsType::Xfs => set_single_disk(answer, udev_info, runtime_info, config), + FsType::Zfs(_) | FsType::Btrfs(_) => { + set_selected_disks(answer, udev_info, runtime_info, config) + } + } +} + +fn set_single_disk( + answer: &Answer, + udev_info: &UdevInfo, + runtime_info: &RuntimeInfo, + config: &mut InstallConfig, +) -> Result<()> { + match &answer.disks.disk_selection { + Some(selection) => { + let disk_name = selection[0].clone(); + let disk = runtime_info + .disks + .iter() + .find(|item| item.path.ends_with(disk_name.as_str())); + match disk { + Some(disk) => config.target_hd = Some(disk.clone()), + None => bail!("disk in 'disk_selection' not found"), + } + } + None => { + let disk_index = + get_single_udev_index(answer.disks.filter.clone().unwrap(), &udev_info.disks)?; + let disk = runtime_info + .disks + .iter() + .find(|item| item.index == disk_index); + config.target_hd = disk.cloned(); + } + } + Ok(()) +} + +fn set_selected_disks( + answer: &Answer, + udev_info: &UdevInfo, + runtime_info: &RuntimeInfo, + config: &mut InstallConfig, +) -> Result<()> { + match &answer.disks.disk_selection { + Some(selection) => { + info!("Disk selection found"); + for disk_name in selection { + let disk = runtime_info + .disks + .iter() + .find(|item| item.path.ends_with(disk_name.as_str())); + if let Some(disk) = disk { + config + .disk_selection + .insert(disk.index.clone(), disk.index.clone()); + } + } + } + None => { + info!("No disk selection found, looking for disk filters"); + let filter_match = answer + .disks + .filter_match + .clone() + .unwrap_or(FilterMatch::Any); + let disk_filters = answer + .disks + .filter + .clone() + .context("no disk filters defined")?; + let selected_disk_indexes = get_matched_udev_indexes( + disk_filters, + &udev_info.disks, + filter_match == FilterMatch::All, + )?; + + for i in selected_disk_indexes.into_iter() { + let disk = runtime_info + .disks + .iter() + .find(|item| item.index == i) + .unwrap(); + config + .disk_selection + .insert(disk.index.clone(), disk.index.clone()); + } + } + } + if config.disk_selection.is_empty() { + bail!("No disks found matching selection."); + } + Ok(()) +} + +pub fn get_first_selected_disk(config: &InstallConfig) -> usize { + config + .disk_selection + .iter() + .next() + .expect("no disks found") + .0 + .parse::<usize>() + .expect("could not parse key to usize") +} + +pub fn verify_locale_settings(answer: &Answer, locales: &LocaleInfo) -> Result<()> { + info!("Verifying locale settings"); + if !locales + .countries + .keys() + .any(|i| i == &answer.global.country) + { + bail!("country code '{}' is not valid", &answer.global.country); + } + if !locales.kmap.keys().any(|i| i == &answer.global.keyboard) { + bail!("keyboard layout '{}' is not valid", &answer.global.keyboard); + } + if !locales + .cczones + .iter() + .any(|(_, zones)| zones.contains(&answer.global.timezone)) + { + bail!("timezone '{}' is not valid", &answer.global.timezone); + } + Ok(()) +} + +pub fn run_cmds(step: &str, cmd_vec: &Option<Vec<String>>) -> Result<()> { + if let Some(cmds) = cmd_vec { + if !cmds.is_empty() { + info!("Running {step}-Commands:"); + run_cmd(cmds)?; + info!("{step}-Commands finished"); + } + } + Ok(()) +} + +fn run_cmd(cmds: &Vec<String>) -> Result<()> { + for cmd in cmds { + info!("Command '{cmd}':"); + let mut child = match Command::new("/bin/bash") + .arg("-c") + .arg(cmd.clone()) + .stdout(Stdio::inherit()) + .stderr(Stdio::inherit()) + .spawn() + { + Ok(child) => child, + Err(err) => bail!("error running command {cmd}: {err}"), + }; + if let Err(err) = child.wait() { + bail!("{err}"); + } + } + + Ok(()) +} + +pub fn parse_answer( + answer: &Answer, + udev_info: &UdevInfo, + runtime_info: &RuntimeInfo, + locales: &LocaleInfo, + setup_info: &SetupInfo, +) -> Result<InstallConfig> { + info!("Parsing answer file"); + info!("Setting File system"); + let filesystem = match &answer.disks.filesystem { + Some(answer::Filesystem::Ext4) => FsType::Ext4, + Some(answer::Filesystem::Xfs) => FsType::Xfs, + Some(answer::Filesystem::ZfsRaid0) => FsType::Zfs(ZfsRaidLevel::Raid0), + Some(answer::Filesystem::ZfsRaid1) => FsType::Zfs(ZfsRaidLevel::Raid1), + Some(answer::Filesystem::ZfsRaid10) => FsType::Zfs(ZfsRaidLevel::Raid10), + Some(answer::Filesystem::ZfsRaidZ1) => FsType::Zfs(ZfsRaidLevel::RaidZ), + Some(answer::Filesystem::ZfsRaidZ2) => FsType::Zfs(ZfsRaidLevel::RaidZ2), + Some(answer::Filesystem::ZfsRaidZ3) => FsType::Zfs(ZfsRaidLevel::RaidZ3), + Some(answer::Filesystem::BtrfsRaid0) => FsType::Btrfs(BtrfsRaidLevel::Raid0), + Some(answer::Filesystem::BtrfsRaid1) => FsType::Btrfs(BtrfsRaidLevel::Raid1), + Some(answer::Filesystem::BtrfsRaid10) => FsType::Btrfs(BtrfsRaidLevel::Raid10), + None => FsType::Ext4, + }; + info!("File system selected: {}", filesystem); + + let network_settings = get_network_settings(answer, udev_info, runtime_info, setup_info)?; + + verify_locale_settings(answer, locales)?; + + let mut config = InstallConfig { + autoreboot: 1_usize, + filesys: filesystem, + hdsize: 0., + swapsize: None, + maxroot: None, + minfree: None, + maxvz: None, + zfs_opts: None, + target_hd: None, + disk_selection: BTreeMap::new(), + + country: answer.global.country.clone(), + timezone: answer.global.timezone.clone(), + keymap: answer.global.keyboard.clone(), + + password: answer.global.password.clone(), + mailto: answer.global.mailto.clone(), + + mngmt_nic: network_settings.ifname, + + hostname: network_settings.fqdn.host().unwrap().to_string(), + domain: network_settings.fqdn.domain(), + cidr: network_settings.address, + gateway: network_settings.gateway, + dns: network_settings.dns_server, + }; + + set_disks(answer, udev_info, runtime_info, &mut config)?; + match &config.filesys { + FsType::Xfs | FsType::Ext4 => { + let lvm = match &answer.disks.lvm { + Some(lvm) => lvm.clone(), + None => answer::LvmOptions::new(), + }; + config.hdsize = lvm.hdsize.unwrap_or(config.target_hd.clone().unwrap().size); + config.swapsize = lvm.swapsize; + config.maxroot = lvm.maxroot; + config.maxvz = lvm.maxvz; + config.minfree = lvm.minfree; + } + FsType::Zfs(_) => { + let zfs = match &answer.disks.zfs { + Some(zfs) => zfs.clone(), + None => answer::ZfsOptions::new(), + }; + let first_selected_disk = get_first_selected_disk(&config); + + config.hdsize = zfs + .hdsize + .unwrap_or(runtime_info.disks[first_selected_disk].size); + config.zfs_opts = Some(InstallZfsOption { + ashift: zfs.ashift.unwrap_or(12), + arc_max: zfs.arc_max.unwrap_or(2048), + compress: match zfs.compress { + Some(answer::ZfsCompressOption::On) => ZfsCompressOption::On, + Some(answer::ZfsCompressOption::Off) => ZfsCompressOption::Off, + Some(answer::ZfsCompressOption::Lzjb) => ZfsCompressOption::Lzjb, + Some(answer::ZfsCompressOption::Lz4) => ZfsCompressOption::Lz4, + Some(answer::ZfsCompressOption::Zle) => ZfsCompressOption::Zle, + Some(answer::ZfsCompressOption::Gzip) => ZfsCompressOption::Gzip, + Some(answer::ZfsCompressOption::Zstd) => ZfsCompressOption::Zstd, + None => ZfsCompressOption::On, + }, + checksum: match zfs.checksum { + Some(answer::ZfsChecksumOption::On) => ZfsChecksumOption::On, + Some(answer::ZfsChecksumOption::Fletcher4) => ZfsChecksumOption::Fletcher4, + Some(answer::ZfsChecksumOption::Sha256) => ZfsChecksumOption::Sha256, + None => ZfsChecksumOption::On, + }, + copies: zfs.copies.unwrap_or(1), + }); + } + FsType::Btrfs(_) => { + let btrfs = match &answer.disks.btrfs { + Some(btrfs) => btrfs.clone(), + None => answer::BtrfsOptions::new(), + }; + let first_selected_disk = get_first_selected_disk(&config); + + config.hdsize = btrfs + .hdsize + .unwrap_or(runtime_info.disks[first_selected_disk].size); + } + } + Ok(config) +} + +#[derive(Clone, Debug, Deserialize, PartialEq)] +#[serde(tag = "type", rename_all = "lowercase")] +pub enum LowLevelMessage { + #[serde(rename = "message")] + Info { + message: String, + }, + Error { + message: String, + }, + Prompt { + query: String, + }, + Finished { + state: String, + message: String, + }, + Progress { + ratio: f32, + text: String, + }, +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn test_glob_patterns() { + let test_value = "foobar"; + assert_eq!(find_with_glob("*bar", test_value), true); + assert_eq!(find_with_glob("foo*", test_value), true); + assert_eq!(find_with_glob("foobar", test_value), true); + assert_eq!(find_with_glob("oobar", test_value), false); + } +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 10/22] auto-installer: add simple logging 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (8 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 09/22] auto-installer: add utils Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 11/22] auto-installer: add tests for answer file parsing Aaron Lauterer ` (13 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Log to stdout and the file the binary needs to set up. This is a first variant. By using the log crate macros we can change that in the future without too much effort. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/Cargo.toml | 2 ++ proxmox-auto-installer/src/lib.rs | 1 + proxmox-auto-installer/src/log.rs | 38 +++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+) create mode 100644 proxmox-auto-installer/src/log.rs diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 67218dd..3dcb5fa 100644 --- a/proxmox-auto-installer/Cargo.toml +++ b/proxmox-auto-installer/Cargo.toml @@ -8,7 +8,9 @@ exclude = [ "build", "debian" ] homepage = "https://www.proxmox.com" [dependencies] +anyhow = "1.0" proxmox-installer-common = { path = "../proxmox-installer-common" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" toml = "0.7" +log = "0.4.20" diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs index 72884c1..6636cc7 100644 --- a/proxmox-auto-installer/src/lib.rs +++ b/proxmox-auto-installer/src/lib.rs @@ -1,3 +1,4 @@ pub mod answer; +pub mod log; pub mod udevinfo; pub mod utils; diff --git a/proxmox-auto-installer/src/log.rs b/proxmox-auto-installer/src/log.rs new file mode 100644 index 0000000..d52912a --- /dev/null +++ b/proxmox-auto-installer/src/log.rs @@ -0,0 +1,38 @@ +use anyhow::{bail, Result}; +use log::{Level, Metadata, Record}; +use std::{fs::File, io::Write, sync::Mutex, sync::OnceLock}; + +pub struct AutoInstLogger; +static LOGFILE: OnceLock<Mutex<File>> = OnceLock::new(); + +impl AutoInstLogger { + pub fn init(path: &str) -> Result<()> { + let f = File::create(path)?; + if LOGFILE.set(Mutex::new(f)).is_err() { + bail!("Cannot set LOGFILE") + } + Ok(()) + } +} + +impl log::Log for AutoInstLogger { + fn enabled(&self, metadata: &Metadata) -> bool { + metadata.level() <= Level::Info + } + + /// Logs to stdout without log level and into log file including log level + fn log(&self, record: &Record) { + if self.enabled(record.metadata()) { + println!("{}", record.args()); + let mut file = LOGFILE + .get() + .expect("could not get LOGFILE") + .lock() + .expect("could not get mutex for LOGFILE"); + file.write_all(format!("{} - {}\n", record.level(), record.args()).as_bytes()) + .expect("could not write to LOGFILE"); + } + } + + fn flush(&self) {} +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 11/22] auto-installer: add tests for answer file parsing 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (9 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 10/22] auto-installer: add simple logging Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 12/22] auto-installer: add auto-installer binary Aaron Lauterer ` (12 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel By matching the resulting json to be passed to the low level installer against known good ones. The environment info was gathered from one of our AMD Epyc Rome test servers to have a realistic starting point. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/tests/parse-answer.rs | 102 ++++++++++++++++++ .../tests/resources/iso-info.json | 1 + .../tests/resources/locales.json | 1 + .../resources/parse_answer/disk_match.json | 29 +++++ .../resources/parse_answer/disk_match.toml | 14 +++ .../parse_answer/disk_match_all.json | 26 +++++ .../parse_answer/disk_match_all.toml | 16 +++ .../parse_answer/disk_match_any.json | 33 ++++++ .../parse_answer/disk_match_any.toml | 16 +++ .../tests/resources/parse_answer/minimal.json | 17 +++ .../tests/resources/parse_answer/minimal.toml | 14 +++ .../resources/parse_answer/nic_matching.json | 17 +++ .../resources/parse_answer/nic_matching.toml | 19 ++++ .../tests/resources/parse_answer/readme | 4 + .../resources/parse_answer/specific_nic.json | 17 +++ .../resources/parse_answer/specific_nic.toml | 19 ++++ .../tests/resources/parse_answer/zfs.json | 27 +++++ .../tests/resources/parse_answer/zfs.toml | 19 ++++ .../tests/resources/run-env-info.json | 1 + .../tests/resources/run-env-udev.json | 1 + 20 files changed, 393 insertions(+) create mode 100644 proxmox-auto-installer/tests/parse-answer.rs create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json create mode 100644 proxmox-auto-installer/tests/resources/locales.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs new file mode 100644 index 0000000..7dabe55 --- /dev/null +++ b/proxmox-auto-installer/tests/parse-answer.rs @@ -0,0 +1,102 @@ +use std::path::PathBuf; + +use serde_json::Value; +use std::fs; + +use proxmox_auto_installer::answer; +use proxmox_auto_installer::answer::Answer; +use proxmox_auto_installer::udevinfo::UdevInfo; +use proxmox_auto_installer::utils::parse_answer; + +use proxmox_installer_common::setup::{ + read_json, RuntimeInfo, LocaleInfo, SetupInfo, +}; + +fn get_test_resource_path() -> Result<PathBuf, String> { + Ok(std::env::current_dir() + .expect("current dir failed") + .join("tests/resources")) +} +fn get_answer(path: PathBuf) -> Result<Answer, String> { + let answer_raw = std::fs::read_to_string(&path).unwrap(); + let answer: answer::Answer = toml::from_str(&answer_raw) + .map_err(|err| format!("error parsing answer.toml: {err}")) + .unwrap(); + + Ok(answer) +} + +fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) { + let installer_info: SetupInfo = { + let mut path = path.clone(); + path.push("iso-info.json"); + + read_json(&path).map_err(|err| format!("Failed to retrieve setup info: {err}")).unwrap() + }; + + let locale_info = { + let mut path = path.clone(); + path.push("locales.json"); + + read_json(&path).map_err(|err| format!("Failed to retrieve locale info: {err}")).unwrap() + }; + + let mut runtime_info: RuntimeInfo = { + let mut path = path.clone(); + path.push("run-env-info.json"); + + read_json(&path) + .map_err(|err| format!("Failed to retrieve runtime environment info: {err}")).unwrap() + }; + + let udev_info: UdevInfo = { + let mut path = path.clone(); + path.push("run-env-udev.json"); + + read_json(&path) + .map_err(|err| format!("Failed to retrieve udev info details: {err}")) + .unwrap() + }; + runtime_info.disks.sort(); + if runtime_info.disks.is_empty() { + panic!("disk list is empty!"); + } + (installer_info, locale_info, runtime_info, udev_info) +} + +#[test] +fn test_parse_answers() { + let path = get_test_resource_path().unwrap(); + let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path); + let mut tests_path = path.clone(); + tests_path.push("parse_answer"); + let test_dir = fs::read_dir(tests_path.clone()).unwrap(); + + for file_entry in test_dir { + let file = file_entry.unwrap(); + if !file.file_type().unwrap().is_file() || file.file_name() == "readme" { + continue; + } + let p = file.path(); + let name = p.file_stem().unwrap().to_str().unwrap(); + let extension = p.extension().unwrap().to_str().unwrap(); + if extension == "toml" { + println!("Test: {name}"); + let answer = get_answer(p.clone()).unwrap(); + let config = &parse_answer(&answer, &udev_info, &runtime_info, &locales, &setup_info).unwrap(); + println!("Selected disks: {:#?}", &config.disk_selection); + let config_json = serde_json::to_string(config); + let config: Value = serde_json::from_str(config_json.unwrap().as_str()).unwrap(); + let mut path = tests_path.clone(); + path.push(format!("{name}.json")); + let compare_raw = std::fs::read_to_string(&path).unwrap(); + let compare: Value = serde_json::from_str(&compare_raw).unwrap(); + if config != compare { + panic!( + "Test {} failed:\nleft: {:#?}\nright: {:#?}\n", + name, config, compare + ); + } + } + } +} diff --git a/proxmox-auto-installer/tests/resources/iso-info.json b/proxmox-auto-installer/tests/resources/iso-info.json new file mode 100644 index 0000000..33cb79b --- /dev/null +++ b/proxmox-auto-installer/tests/resources/iso-info.json @@ -0,0 +1 @@ +{"iso-info":{"isoname":"proxmox-ve","isorelease":"2","product":"pve","productlong":"Proxmox VE","release":"8.0"},"locations":{"iso":"/cdrom","lib":"/var/lib/proxmox-installer","pkg":"/cdrom/proxmox/packages/","run":"/run/proxmox-installer"},"product":"pve","product-cfg":{"bridged_network":1,"enable_btrfs":1,"fullname":"Proxmox VE","port":"8006","product":"pve"},"run-env-cache-file":"/run/proxmox-installer/run-env-info.json"} diff --git a/proxmox-auto-installer/tests/resources/locales.json b/proxmox-auto-installer/tests/resources/locales.json new file mode 100644 index 0000000..220a18c --- /dev/null +++ b/proxmox-auto-installer/tests/resources/locales.json @@ -0,0 +1 @@ +{"cczones":{"ad":{"Europe/Andorra":1},"ae":{"Asia/Dubai":1},"af":{"Asia/Kabul":1},"ag":{"America/Antigua":1},"ai":{"America/Anguilla":1},"al":{"Europe/Tirane":1},"am":{"Asia/Yerevan":1},"ao":{"Africa/Luanda":1},"aq":{"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1},"ar":{"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1},"as":{"Pacific/Pago_Pago":1},"at":{"Europe/Vienna":1},"au":{"Antarctica/Macquarie":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1},"aw":{"America/Aruba":1},"ax":{"Europe/Mariehamn":1},"az":{"Asia/Baku":1},"ba":{"Europe/Sarajevo":1},"bb":{"America/Barbados":1},"bd":{"Asia/Dhaka":1},"be":{"Europe/Brussels":1},"bf":{"Africa/Ouagadougou":1},"bg":{"Europe/Sofia":1},"bh":{"Asia/Bahrain":1},"bi":{"Africa/Bujumbura":1},"bj":{"Africa/Porto-Novo":1},"bl":{"America/St_Barthelemy":1},"bm":{"Atlantic/Bermuda":1},"bn":{"Asia/Brunei":1},"bo":{"America/La_Paz":1},"bq":{"America/Kralendijk":1},"br":{"America/Araguaina":1,"America/Bahia":1,"America/Belem":1,"America/Boa_Vista":1,"America/Campo_Grande":1,"America/Cuiaba":1,"America/Eirunepe":1,"America/Fortaleza":1,"America/Maceio":1,"America/Manaus":1,"America/Noronha":1,"America/Porto_Velho":1,"America/Recife":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Sao_Paulo":1},"bs":{"America/Nassau":1},"bt":{"Asia/Thimphu":1},"bw":{"Africa/Gaborone":1},"by":{"Europe/Minsk":1},"bz":{"America/Belize":1},"ca":{"America/Atikokan":1,"America/Blanc-Sablon":1,"America/Cambridge_Bay":1,"America/Creston":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Edmonton":1,"America/Fort_Nelson":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Halifax":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Moncton":1,"America/Rankin_Inlet":1,"America/Regina":1,"America/Resolute":1,"America/St_Johns":1,"America/Swift_Current":1,"America/Toronto":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1},"cc":{"Indian/Cocos":1},"cd":{"Africa/Kinshasa":1,"Africa/Lubumbashi":1},"cf":{"Africa/Bangui":1},"cg":{"Africa/Brazzaville":1},"ch":{"Europe/Zurich":1},"ci":{"Africa/Abidjan":1},"ck":{"Pacific/Rarotonga":1},"cl":{"America/Punta_Arenas":1,"America/Santiago":1,"Pacific/Easter":1},"cm":{"Africa/Douala":1},"cn":{"Asia/Shanghai":1,"Asia/Urumqi":1},"co":{"America/Bogota":1},"cr":{"America/Costa_Rica":1},"cu":{"America/Havana":1},"cv":{"Atlantic/Cape_Verde":1},"cw":{"America/Curacao":1},"cx":{"Indian/Christmas":1},"cy":{"Asia/Famagusta":1,"Asia/Nicosia":1},"cz":{"Europe/Prague":1},"de":{"Europe/Berlin":1,"Europe/Busingen":1},"dj":{"Africa/Djibouti":1},"dk":{"Europe/Copenhagen":1},"dm":{"America/Dominica":1},"do":{"America/Santo_Domingo":1},"dz":{"Africa/Algiers":1},"ec":{"America/Guayaquil":1,"Pacific/Galapagos":1},"ee":{"Europe/Tallinn":1},"eg":{"Africa/Cairo":1},"eh":{"Africa/El_Aaiun":1},"er":{"Africa/Asmara":1},"es":{"Africa/Ceuta":1,"Atlantic/Canary":1,"Europe/Madrid":1},"et":{"Africa/Addis_Ababa":1},"fi":{"Europe/Helsinki":1},"fj":{"Pacific/Fiji":1},"fk":{"Atlantic/Stanley":1},"fm":{"Pacific/Chuuk":1,"Pacific/Kosrae":1,"Pacific/Pohnpei":1},"fo":{"Atlantic/Faroe":1},"fr":{"Europe/Paris":1},"ga":{"Africa/Libreville":1},"gb":{"Europe/London":1},"gd":{"America/Grenada":1},"ge":{"Asia/Tbilisi":1},"gf":{"America/Cayenne":1},"gg":{"Europe/Guernsey":1},"gh":{"Africa/Accra":1},"gi":{"Europe/Gibraltar":1},"gl":{"America/Danmarkshavn":1,"America/Nuuk":1,"America/Scoresbysund":1,"America/Thule":1},"gm":{"Africa/Banjul":1},"gn":{"Africa/Conakry":1},"gp":{"America/Guadeloupe":1},"gq":{"Africa/Malabo":1},"gr":{"Europe/Athens":1},"gs":{"Atlantic/South_Georgia":1},"gt":{"America/Guatemala":1},"gu":{"Pacific/Guam":1},"gw":{"Africa/Bissau":1},"gy":{"America/Guyana":1},"hk":{"Asia/Hong_Kong":1},"hn":{"America/Tegucigalpa":1},"hr":{"Europe/Zagreb":1},"ht":{"America/Port-au-Prince":1},"hu":{"Europe/Budapest":1},"id":{"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Makassar":1,"Asia/Pontianak":1},"ie":{"Europe/Dublin":1},"il":{"Asia/Jerusalem":1},"im":{"Europe/Isle_of_Man":1},"in":{"Asia/Kolkata":1},"io":{"Indian/Chagos":1},"iq":{"Asia/Baghdad":1},"ir":{"Asia/Tehran":1},"is":{"Atlantic/Reykjavik":1},"it":{"Europe/Rome":1},"je":{"Europe/Jersey":1},"jm":{"America/Jamaica":1},"jo":{"Asia/Amman":1},"jp":{"Asia/Tokyo":1},"ke":{"Africa/Nairobi":1},"kg":{"Asia/Bishkek":1},"kh":{"Asia/Phnom_Penh":1},"ki":{"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Tarawa":1},"km":{"Indian/Comoro":1},"kn":{"America/St_Kitts":1},"kp":{"Asia/Pyongyang":1},"kr":{"Asia/Seoul":1},"kw":{"Asia/Kuwait":1},"ky":{"America/Cayman":1},"kz":{"Asia/Almaty":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Atyrau":1,"Asia/Oral":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1},"la":{"Asia/Vientiane":1},"lb":{"Asia/Beirut":1},"lc":{"America/St_Lucia":1},"li":{"Europe/Vaduz":1},"lk":{"Asia/Colombo":1},"lr":{"Africa/Monrovia":1},"ls":{"Africa/Maseru":1},"lt":{"Europe/Vilnius":1},"lu":{"Europe/Luxembourg":1},"lv":{"Europe/Riga":1},"ly":{"Africa/Tripoli":1},"ma":{"Africa/Casablanca":1},"mc":{"Europe/Monaco":1},"md":{"Europe/Chisinau":1},"me":{"Europe/Podgorica":1},"mf":{"America/Marigot":1},"mg":{"Indian/Antananarivo":1},"mh":{"Pacific/Kwajalein":1,"Pacific/Majuro":1},"mk":{"Europe/Skopje":1},"ml":{"Africa/Bamako":1},"mm":{"Asia/Yangon":1},"mn":{"Asia/Choibalsan":1,"Asia/Hovd":1,"Asia/Ulaanbaatar":1},"mo":{"Asia/Macau":1},"mp":{"Pacific/Saipan":1},"mq":{"America/Martinique":1},"mr":{"Africa/Nouakchott":1},"ms":{"America/Montserrat":1},"mt":{"Europe/Malta":1},"mu":{"Indian/Mauritius":1},"mv":{"Indian/Maldives":1},"mw":{"Africa/Blantyre":1},"mx":{"America/Bahia_Banderas":1,"America/Cancun":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Hermosillo":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Merida":1,"America/Mexico_City":1,"America/Monterrey":1,"America/Ojinaga":1,"America/Tijuana":1},"my":{"Asia/Kuala_Lumpur":1,"Asia/Kuching":1},"mz":{"Africa/Maputo":1},"na":{"Africa/Windhoek":1},"nc":{"Pacific/Noumea":1},"ne":{"Africa/Niamey":1},"nf":{"Pacific/Norfolk":1},"ng":{"Africa/Lagos":1},"ni":{"America/Managua":1},"nl":{"Europe/Amsterdam":1},"no":{"Europe/Oslo":1},"np":{"Asia/Kathmandu":1},"nr":{"Pacific/Nauru":1},"nu":{"Pacific/Niue":1},"nz":{"Pacific/Auckland":1,"Pacific/Chatham":1},"om":{"Asia/Muscat":1},"pa":{"America/Panama":1},"pe":{"America/Lima":1},"pf":{"Pacific/Gambier":1,"Pacific/Marquesas":1,"Pacific/Tahiti":1},"pg":{"Pacific/Bougainville":1,"Pacific/Port_Moresby":1},"ph":{"Asia/Manila":1},"pk":{"Asia/Karachi":1},"pl":{"Europe/Warsaw":1},"pm":{"America/Miquelon":1},"pn":{"Pacific/Pitcairn":1},"pr":{"America/Puerto_Rico":1},"ps":{"Asia/Gaza":1,"Asia/Hebron":1},"pt":{"Atlantic/Azores":1,"Atlantic/Madeira":1,"Europe/Lisbon":1},"pw":{"Pacific/Palau":1},"py":{"America/Asuncion":1},"qa":{"Asia/Qatar":1},"re":{"Indian/Reunion":1},"ro":{"Europe/Bucharest":1},"rs":{"Europe/Belgrade":1},"ru":{"Asia/Anadyr":1,"Asia/Barnaul":1,"Asia/Chita":1,"Asia/Irkutsk":1,"Asia/Kamchatka":1,"Asia/Khandyga":1,"Asia/Krasnoyarsk":1,"Asia/Magadan":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Sakhalin":1,"Asia/Srednekolymsk":1,"Asia/Tomsk":1,"Asia/Ust-Nera":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yekaterinburg":1,"Europe/Astrakhan":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Moscow":1,"Europe/Samara":1,"Europe/Saratov":1,"Europe/Ulyanovsk":1,"Europe/Volgograd":1},"rw":{"Africa/Kigali":1},"sa":{"Asia/Riyadh":1},"sb":{"Pacific/Guadalcanal":1},"sc":{"Indian/Mahe":1},"sd":{"Africa/Khartoum":1},"se":{"Europe/Stockholm":1},"sg":{"Asia/Singapore":1},"sh":{"Atlantic/St_Helena":1},"si":{"Europe/Ljubljana":1},"sj":{"Arctic/Longyearbyen":1},"sk":{"Europe/Bratislava":1},"sl":{"Africa/Freetown":1},"sm":{"Europe/San_Marino":1},"sn":{"Africa/Dakar":1},"so":{"Africa/Mogadishu":1},"sr":{"America/Paramaribo":1},"ss":{"Africa/Juba":1},"st":{"Africa/Sao_Tome":1},"sv":{"America/El_Salvador":1},"sx":{"America/Lower_Princes":1},"sy":{"Asia/Damascus":1},"sz":{"Africa/Mbabane":1},"tc":{"America/Grand_Turk":1},"td":{"Africa/Ndjamena":1},"tf":{"Indian/Kerguelen":1},"tg":{"Africa/Lome":1},"th":{"Asia/Bangkok":1},"tj":{"Asia/Dushanbe":1},"tk":{"Pacific/Fakaofo":1},"tl":{"Asia/Dili":1},"tm":{"Asia/Ashgabat":1},"tn":{"Africa/Tunis":1},"to":{"Pacific/Tongatapu":1},"tr":{"Europe/Istanbul":1},"tt":{"America/Port_of_Spain":1},"tv":{"Pacific/Funafuti":1},"tw":{"Asia/Taipei":1},"tz":{"Africa/Dar_es_Salaam":1},"ua":{"Europe/Kyiv":1,"Europe/Simferopol":1},"ug":{"Africa/Kampala":1},"um":{"Pacific/Midway":1,"Pacific/Wake":1},"us":{"America/Adak":1,"America/Anchorage":1,"America/Boise":1,"America/Chicago":1,"America/Denver":1,"America/Detroit":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Los_Angeles":1,"America/Menominee":1,"America/Metlakatla":1,"America/New_York":1,"America/Nome":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Phoenix":1,"America/Sitka":1,"America/Yakutat":1,"Pacific/Honolulu":1},"uy":{"America/Montevideo":1},"uz":{"Asia/Samarkand":1,"Asia/Tashkent":1},"va":{"Europe/Vatican":1},"vc":{"America/St_Vincent":1},"ve":{"America/Caracas":1},"vg":{"America/Tortola":1},"vi":{"America/St_Thomas":1},"vn":{"Asia/Ho_Chi_Minh":1},"vu":{"Pacific/Efate":1},"wf":{"Pacific/Wallis":1},"ws":{"Pacific/Apia":1},"ye":{"Asia/Aden":1},"yt":{"Indian/Mayotte":1},"za":{"Africa/Johannesburg":1},"zm":{"Africa/Lusaka":1},"zw":{"Africa/Harare":1}},"country":{"ad":{"kmap":"","mirror":"","name":"Andorra","zone":"Europe/Andorra"},"ae":{"kmap":"","mirror":"","name":"United Arab Emirates","zone":"Asia/Dubai"},"af":{"kmap":"","mirror":"","name":"Afghanistan","zone":"Asia/Kabul"},"ag":{"kmap":"","mirror":"","name":"Antigua and Barbuda","zone":"America/Antigua"},"ai":{"kmap":"","mirror":"","name":"Anguilla","zone":"America/Anguilla"},"al":{"kmap":"","mirror":"","name":"Albania","zone":"Europe/Tirane"},"am":{"kmap":"","mirror":"","name":"Armenia","zone":"Asia/Yerevan"},"ao":{"kmap":"","mirror":"","name":"Angola","zone":"Africa/Luanda"},"aq":{"kmap":"","mirror":"","name":"Antarctica","zone":"Antarctica/McMurdo"},"ar":{"kmap":"","mirror":"","name":"Argentina","zone":"America/Argentina/Buenos_Aires"},"as":{"kmap":"","mirror":"","name":"American Samoa","zone":"Pacific/Pago_Pago"},"at":{"kmap":"de","mirror":"ftp.at.debian.org","name":"Austria","zone":"Europe/Vienna"},"au":{"kmap":"","mirror":"ftp.au.debian.org","name":"Australia","zone":"Australia/Lord_Howe"},"aw":{"kmap":"","mirror":"","name":"Aruba","zone":"America/Aruba"},"ax":{"kmap":"","mirror":"","name":"Åland Islands","zone":"Europe/Mariehamn"},"az":{"kmap":"","mirror":"","name":"Azerbaijan","zone":"Asia/Baku"},"ba":{"kmap":"","mirror":"","name":"Bosnia and Herzegovina","zone":"Europe/Sarajevo"},"bb":{"kmap":"","mirror":"","name":"Barbados","zone":"America/Barbados"},"bd":{"kmap":"","mirror":"","name":"Bangladesh","zone":"Asia/Dhaka"},"be":{"kmap":"fr-be","mirror":"ftp.be.debian.org","name":"Belgium","zone":"Europe/Brussels"},"bf":{"kmap":"","mirror":"","name":"Burkina Faso","zone":"Africa/Ouagadougou"},"bg":{"kmap":"","mirror":"ftp.bg.debian.org","name":"Bulgaria","zone":"Europe/Sofia"},"bh":{"kmap":"","mirror":"","name":"Bahrain","zone":"Asia/Bahrain"},"bi":{"kmap":"","mirror":"","name":"Burundi","zone":"Africa/Bujumbura"},"bj":{"kmap":"","mirror":"","name":"Benin","zone":"Africa/Porto-Novo"},"bl":{"kmap":"","mirror":"","name":"Saint Barthélemy","zone":"America/St_Barthelemy"},"bm":{"kmap":"","mirror":"","name":"Bermuda","zone":"Atlantic/Bermuda"},"bn":{"kmap":"","mirror":"","name":"Brunei Darussalam","zone":"Asia/Brunei"},"bo":{"kmap":"","mirror":"","name":"Bolivia","zone":"America/La_Paz"},"bq":{"kmap":"","mirror":"","name":"Bonaire, Sint Eustatius and Saba","zone":"America/Kralendijk"},"br":{"kmap":"pt-br","mirror":"ftp.br.debian.org","name":"Brazil","zone":"America/Noronha"},"bs":{"kmap":"","mirror":"","name":"Bahamas","zone":"America/Nassau"},"bt":{"kmap":"","mirror":"","name":"Bhutan","zone":"Asia/Thimphu"},"bv":{"kmap":"","mirror":"","name":"Bouvet Island"},"bw":{"kmap":"","mirror":"","name":"Botswana","zone":"Africa/Gaborone"},"by":{"kmap":"","mirror":"","name":"Belarus","zone":"Europe/Minsk"},"bz":{"kmap":"","mirror":"","name":"Belize","zone":"America/Belize"},"ca":{"kmap":"en-us","mirror":"ftp.ca.debian.org","name":"Canada","zone":"America/St_Johns"},"cc":{"kmap":"","mirror":"","name":"Cocos (Keeling) Islands","zone":"Indian/Cocos"},"cd":{"kmap":"","mirror":"","name":"Congo, The Democratic Republic of the","zone":"Africa/Kinshasa"},"cf":{"kmap":"","mirror":"","name":"Central African Republic","zone":"Africa/Bangui"},"cg":{"kmap":"","mirror":"","name":"Congo","zone":"Africa/Brazzaville"},"ch":{"kmap":"de-ch","mirror":"ftp.ch.debian.org","name":"Switzerland","zone":"Europe/Zurich"},"ci":{"kmap":"","mirror":"","name":"Côte d'Ivoire","zone":"Africa/Abidjan"},"ck":{"kmap":"","mirror":"","name":"Cook Islands","zone":"Pacific/Rarotonga"},"cl":{"kmap":"","mirror":"ftp.cl.debian.org","name":"Chile","zone":"America/Santiago"},"cm":{"kmap":"","mirror":"","name":"Cameroon","zone":"Africa/Douala"},"cn":{"kmap":"","mirror":"","name":"China","zone":"Asia/Shanghai"},"co":{"kmap":"","mirror":"","name":"Colombia","zone":"America/Bogota"},"cr":{"kmap":"","mirror":"","name":"Costa Rica","zone":"America/Costa_Rica"},"cu":{"kmap":"","mirror":"","name":"Cuba","zone":"America/Havana"},"cv":{"kmap":"","mirror":"","name":"Cabo Verde","zone":"Atlantic/Cape_Verde"},"cw":{"kmap":"","mirror":"","name":"Curaçao","zone":"America/Curacao"},"cx":{"kmap":"","mirror":"","name":"Christmas Island","zone":"Indian/Christmas"},"cy":{"kmap":"","mirror":"","name":"Cyprus","zone":"Asia/Nicosia"},"cz":{"kmap":"","mirror":"ftp.cz.debian.org","name":"Czechia","zone":"Europe/Prague"},"de":{"kmap":"de","mirror":"ftp.de.debian.org","name":"Germany","zone":"Europe/Berlin"},"dj":{"kmap":"","mirror":"","name":"Djibouti","zone":"Africa/Djibouti"},"dk":{"kmap":"dk","mirror":"ftp.dk.debian.org","name":"Denmark","zone":"Europe/Copenhagen"},"dm":{"kmap":"","mirror":"","name":"Dominica","zone":"America/Dominica"},"do":{"kmap":"","mirror":"","name":"Dominican Republic","zone":"America/Santo_Domingo"},"dz":{"kmap":"","mirror":"","name":"Algeria","zone":"Africa/Algiers"},"ec":{"kmap":"","mirror":"","name":"Ecuador","zone":"America/Guayaquil"},"ee":{"kmap":"","mirror":"ftp.ee.debian.org","name":"Estonia","zone":"Europe/Tallinn"},"eg":{"kmap":"","mirror":"","name":"Egypt","zone":"Africa/Cairo"},"eh":{"kmap":"","mirror":"","name":"Western Sahara","zone":"Africa/El_Aaiun"},"er":{"kmap":"","mirror":"","name":"Eritrea","zone":"Africa/Asmara"},"es":{"kmap":"es","mirror":"ftp.es.debian.org","name":"Spain","zone":"Europe/Madrid"},"et":{"kmap":"","mirror":"","name":"Ethiopia","zone":"Africa/Addis_Ababa"},"fi":{"kmap":"fi","mirror":"ftp.fi.debian.org","name":"Finland","zone":"Europe/Helsinki"},"fj":{"kmap":"","mirror":"","name":"Fiji","zone":"Pacific/Fiji"},"fk":{"kmap":"","mirror":"","name":"Falkland Islands (Malvinas)","zone":"Atlantic/Stanley"},"fm":{"kmap":"","mirror":"","name":"Micronesia, Federated States of","zone":"Pacific/Chuuk"},"fo":{"kmap":"","mirror":"","name":"Faroe Islands","zone":"Atlantic/Faroe"},"fr":{"kmap":"fr","mirror":"ftp.fr.debian.org","name":"France","zone":"Europe/Paris"},"ga":{"kmap":"","mirror":"","name":"Gabon","zone":"Africa/Libreville"},"gb":{"kmap":"en-gb","mirror":"ftp.uk.debian.org","name":"United Kingdom","zone":"Europe/London"},"gd":{"kmap":"","mirror":"","name":"Grenada","zone":"America/Grenada"},"ge":{"kmap":"","mirror":"","name":"Georgia","zone":"Asia/Tbilisi"},"gf":{"kmap":"","mirror":"","name":"French Guiana","zone":"America/Cayenne"},"gg":{"kmap":"","mirror":"","name":"Guernsey","zone":"Europe/Guernsey"},"gh":{"kmap":"","mirror":"","name":"Ghana","zone":"Africa/Accra"},"gi":{"kmap":"es","mirror":"","name":"Gibraltar","zone":"Europe/Gibraltar"},"gl":{"kmap":"","mirror":"","name":"Greenland","zone":"America/Nuuk"},"gm":{"kmap":"","mirror":"","name":"Gambia","zone":"Africa/Banjul"},"gn":{"kmap":"","mirror":"","name":"Guinea","zone":"Africa/Conakry"},"gp":{"kmap":"","mirror":"","name":"Guadeloupe","zone":"America/Guadeloupe"},"gq":{"kmap":"","mirror":"","name":"Equatorial Guinea","zone":"Africa/Malabo"},"gr":{"kmap":"","mirror":"ftp.gr.debian.org","name":"Greece","zone":"Europe/Athens"},"gs":{"kmap":"","mirror":"","name":"South Georgia and the South Sandwich Islands","zone":"Atlantic/South_Georgia"},"gt":{"kmap":"","mirror":"","name":"Guatemala","zone":"America/Guatemala"},"gu":{"kmap":"","mirror":"","name":"Guam","zone":"Pacific/Guam"},"gw":{"kmap":"","mirror":"","name":"Guinea-Bissau","zone":"Africa/Bissau"},"gy":{"kmap":"","mirror":"","name":"Guyana","zone":"America/Guyana"},"hk":{"kmap":"","mirror":"ftp.hk.debian.org","name":"Hong Kong","zone":"Asia/Hong_Kong"},"hm":{"kmap":"","mirror":"","name":"Heard Island and McDonald Islands"},"hn":{"kmap":"","mirror":"","name":"Honduras","zone":"America/Tegucigalpa"},"hr":{"kmap":"","mirror":"ftp.hr.debian.org","name":"Croatia","zone":"Europe/Zagreb"},"ht":{"kmap":"","mirror":"","name":"Haiti","zone":"America/Port-au-Prince"},"hu":{"kmap":"hu","mirror":"ftp.hu.debian.org","name":"Hungary","zone":"Europe/Budapest"},"id":{"kmap":"","mirror":"","name":"Indonesia","zone":"Asia/Jakarta"},"ie":{"kmap":"","mirror":"ftp.ie.debian.org","name":"Ireland","zone":"Europe/Dublin"},"il":{"kmap":"","mirror":"","name":"Israel","zone":"Asia/Jerusalem"},"im":{"kmap":"","mirror":"","name":"Isle of Man","zone":"Europe/Isle_of_Man"},"in":{"kmap":"","mirror":"","name":"India","zone":"Asia/Kolkata"},"io":{"kmap":"","mirror":"","name":"British Indian Ocean Territory","zone":"Indian/Chagos"},"iq":{"kmap":"","mirror":"","name":"Iraq","zone":"Asia/Baghdad"},"ir":{"kmap":"","mirror":"","name":"Iran","zone":"Asia/Tehran"},"is":{"kmap":"is","mirror":"ftp.is.debian.org","name":"Iceland","zone":"Atlantic/Reykjavik"},"it":{"kmap":"it","mirror":"ftp.it.debian.org","name":"Italy","zone":"Europe/Rome"},"je":{"kmap":"","mirror":"","name":"Jersey","zone":"Europe/Jersey"},"jm":{"kmap":"","mirror":"","name":"Jamaica","zone":"America/Jamaica"},"jo":{"kmap":"","mirror":"","name":"Jordan","zone":"Asia/Amman"},"jp":{"kmap":"jp","mirror":"ftp.jp.debian.org","name":"Japan","zone":"Asia/Tokyo"},"ke":{"kmap":"","mirror":"","name":"Kenya","zone":"Africa/Nairobi"},"kg":{"kmap":"","mirror":"","name":"Kyrgyzstan","zone":"Asia/Bishkek"},"kh":{"kmap":"","mirror":"","name":"Cambodia","zone":"Asia/Phnom_Penh"},"ki":{"kmap":"","mirror":"","name":"Kiribati","zone":"Pacific/Tarawa"},"km":{"kmap":"","mirror":"","name":"Comoros","zone":"Indian/Comoro"},"kn":{"kmap":"","mirror":"","name":"Saint Kitts and Nevis","zone":"America/St_Kitts"},"kp":{"kmap":"","mirror":"","name":"North Korea","zone":"Asia/Pyongyang"},"kr":{"kmap":"","mirror":"ftp.kr.debian.org","name":"South Korea","zone":"Asia/Seoul"},"kw":{"kmap":"","mirror":"","name":"Kuwait","zone":"Asia/Kuwait"},"ky":{"kmap":"","mirror":"","name":"Cayman Islands","zone":"America/Cayman"},"kz":{"kmap":"","mirror":"","name":"Kazakhstan","zone":"Asia/Almaty"},"la":{"kmap":"","mirror":"","name":"Laos","zone":"Asia/Vientiane"},"lb":{"kmap":"","mirror":"","name":"Lebanon","zone":"Asia/Beirut"},"lc":{"kmap":"","mirror":"","name":"Saint Lucia","zone":"America/St_Lucia"},"li":{"kmap":"de-ch","mirror":"","name":"Liechtenstein","zone":"Europe/Vaduz"},"lk":{"kmap":"","mirror":"","name":"Sri Lanka","zone":"Asia/Colombo"},"lr":{"kmap":"","mirror":"","name":"Liberia","zone":"Africa/Monrovia"},"ls":{"kmap":"","mirror":"","name":"Lesotho","zone":"Africa/Maseru"},"lt":{"kmap":"lt","mirror":"","name":"Lithuania","zone":"Europe/Vilnius"},"lu":{"kmap":"fr-ch","mirror":"","name":"Luxembourg","zone":"Europe/Luxembourg"},"lv":{"kmap":"","mirror":"","name":"Latvia","zone":"Europe/Riga"},"ly":{"kmap":"","mirror":"","name":"Libya","zone":"Africa/Tripoli"},"ma":{"kmap":"","mirror":"","name":"Morocco","zone":"Africa/Casablanca"},"mc":{"kmap":"","mirror":"","name":"Monaco","zone":"Europe/Monaco"},"md":{"kmap":"","mirror":"","name":"Moldova","zone":"Europe/Chisinau"},"me":{"kmap":"","mirror":"","name":"Montenegro","zone":"Europe/Podgorica"},"mf":{"kmap":"","mirror":"","name":"Saint Martin (French part)","zone":"America/Marigot"},"mg":{"kmap":"","mirror":"","name":"Madagascar","zone":"Indian/Antananarivo"},"mh":{"kmap":"","mirror":"","name":"Marshall Islands","zone":"Pacific/Majuro"},"mk":{"kmap":"mk","mirror":"","name":"North Macedonia","zone":"Europe/Skopje"},"ml":{"kmap":"","mirror":"","name":"Mali","zone":"Africa/Bamako"},"mm":{"kmap":"","mirror":"","name":"Myanmar","zone":"Asia/Yangon"},"mn":{"kmap":"","mirror":"","name":"Mongolia","zone":"Asia/Ulaanbaatar"},"mo":{"kmap":"","mirror":"","name":"Macao","zone":"Asia/Macau"},"mp":{"kmap":"","mirror":"","name":"Northern Mariana Islands","zone":"Pacific/Saipan"},"mq":{"kmap":"","mirror":"","name":"Martinique","zone":"America/Martinique"},"mr":{"kmap":"","mirror":"","name":"Mauritania","zone":"Africa/Nouakchott"},"ms":{"kmap":"","mirror":"","name":"Montserrat","zone":"America/Montserrat"},"mt":{"kmap":"","mirror":"","name":"Malta","zone":"Europe/Malta"},"mu":{"kmap":"","mirror":"","name":"Mauritius","zone":"Indian/Mauritius"},"mv":{"kmap":"","mirror":"","name":"Maldives","zone":"Indian/Maldives"},"mw":{"kmap":"","mirror":"","name":"Malawi","zone":"Africa/Blantyre"},"mx":{"kmap":"","mirror":"ftp.mx.debian.org","name":"Mexico","zone":"America/Mexico_City"},"my":{"kmap":"","mirror":"","name":"Malaysia","zone":"Asia/Kuala_Lumpur"},"mz":{"kmap":"","mirror":"","name":"Mozambique","zone":"Africa/Maputo"},"na":{"kmap":"","mirror":"","name":"Namibia","zone":"Africa/Windhoek"},"nc":{"kmap":"","mirror":"","name":"New Caledonia","zone":"Pacific/Noumea"},"ne":{"kmap":"","mirror":"","name":"Niger","zone":"Africa/Niamey"},"nf":{"kmap":"","mirror":"","name":"Norfolk Island","zone":"Pacific/Norfolk"},"ng":{"kmap":"","mirror":"","name":"Nigeria","zone":"Africa/Lagos"},"ni":{"kmap":"","mirror":"","name":"Nicaragua","zone":"America/Managua"},"nl":{"kmap":"en-us","mirror":"ftp.nl.debian.org","name":"Netherlands","zone":"Europe/Amsterdam"},"no":{"kmap":"no","mirror":"ftp.no.debian.org","name":"Norway","zone":"Europe/Oslo"},"np":{"kmap":"","mirror":"","name":"Nepal","zone":"Asia/Kathmandu"},"nr":{"kmap":"","mirror":"","name":"Nauru","zone":"Pacific/Nauru"},"nu":{"kmap":"","mirror":"","name":"Niue","zone":"Pacific/Niue"},"nz":{"kmap":"","mirror":"ftp.nz.debian.org","name":"New Zealand","zone":"Pacific/Auckland"},"om":{"kmap":"","mirror":"","name":"Oman","zone":"Asia/Muscat"},"pa":{"kmap":"","mirror":"","name":"Panama","zone":"America/Panama"},"pe":{"kmap":"","mirror":"","name":"Peru","zone":"America/Lima"},"pf":{"kmap":"","mirror":"","name":"French Polynesia","zone":"Pacific/Tahiti"},"pg":{"kmap":"","mirror":"","name":"Papua New Guinea","zone":"Pacific/Port_Moresby"},"ph":{"kmap":"","mirror":"","name":"Philippines","zone":"Asia/Manila"},"pk":{"kmap":"","mirror":"","name":"Pakistan","zone":"Asia/Karachi"},"pl":{"kmap":"pl","mirror":"ftp.pl.debian.org","name":"Poland","zone":"Europe/Warsaw"},"pm":{"kmap":"","mirror":"","name":"Saint Pierre and Miquelon","zone":"America/Miquelon"},"pn":{"kmap":"","mirror":"","name":"Pitcairn","zone":"Pacific/Pitcairn"},"pr":{"kmap":"","mirror":"","name":"Puerto Rico","zone":"America/Puerto_Rico"},"ps":{"kmap":"","mirror":"","name":"Palestine, State of","zone":"Asia/Gaza"},"pt":{"kmap":"pt","mirror":"ftp.pt.debian.org","name":"Portugal","zone":"Europe/Lisbon"},"pw":{"kmap":"","mirror":"","name":"Palau","zone":"Pacific/Palau"},"py":{"kmap":"","mirror":"","name":"Paraguay","zone":"America/Asuncion"},"qa":{"kmap":"","mirror":"","name":"Qatar","zone":"Asia/Qatar"},"re":{"kmap":"","mirror":"","name":"Réunion","zone":"Indian/Reunion"},"ro":{"kmap":"","mirror":"ftp.ro.debian.org","name":"Romania","zone":"Europe/Bucharest"},"rs":{"kmap":"","mirror":"","name":"Serbia","zone":"Europe/Belgrade"},"ru":{"kmap":"","mirror":"ftp.ru.debian.org","name":"Russian Federation","zone":"Europe/Kaliningrad"},"rw":{"kmap":"","mirror":"","name":"Rwanda","zone":"Africa/Kigali"},"sa":{"kmap":"","mirror":"","name":"Saudi Arabia","zone":"Asia/Riyadh"},"sb":{"kmap":"","mirror":"","name":"Solomon Islands","zone":"Pacific/Guadalcanal"},"sc":{"kmap":"","mirror":"","name":"Seychelles","zone":"Indian/Mahe"},"sd":{"kmap":"","mirror":"","name":"Sudan","zone":"Africa/Khartoum"},"se":{"kmap":"","mirror":"ftp.se.debian.org","name":"Sweden","zone":"Europe/Stockholm"},"sg":{"kmap":"","mirror":"","name":"Singapore","zone":"Asia/Singapore"},"sh":{"kmap":"","mirror":"","name":"Saint Helena, Ascension and Tristan da Cunha","zone":"Atlantic/St_Helena"},"si":{"kmap":"si","mirror":"ftp.si.debian.org","name":"Slovenia","zone":"Europe/Ljubljana"},"sj":{"kmap":"","mirror":"","name":"Svalbard and Jan Mayen","zone":"Arctic/Longyearbyen"},"sk":{"kmap":"","mirror":"ftp.sk.debian.org","name":"Slovakia","zone":"Europe/Bratislava"},"sl":{"kmap":"","mirror":"","name":"Sierra Leone","zone":"Africa/Freetown"},"sm":{"kmap":"","mirror":"","name":"San Marino","zone":"Europe/San_Marino"},"sn":{"kmap":"","mirror":"","name":"Senegal","zone":"Africa/Dakar"},"so":{"kmap":"","mirror":"","name":"Somalia","zone":"Africa/Mogadishu"},"sr":{"kmap":"","mirror":"","name":"Suriname","zone":"America/Paramaribo"},"ss":{"kmap":"","mirror":"","name":"South Sudan","zone":"Africa/Juba"},"st":{"kmap":"","mirror":"","name":"Sao Tome and Principe","zone":"Africa/Sao_Tome"},"sv":{"kmap":"","mirror":"","name":"El Salvador","zone":"America/El_Salvador"},"sx":{"kmap":"","mirror":"","name":"Sint Maarten (Dutch part)","zone":"America/Lower_Princes"},"sy":{"kmap":"","mirror":"","name":"Syria","zone":"Asia/Damascus"},"sz":{"kmap":"","mirror":"","name":"Eswatini","zone":"Africa/Mbabane"},"tc":{"kmap":"","mirror":"","name":"Turks and Caicos Islands","zone":"America/Grand_Turk"},"td":{"kmap":"","mirror":"","name":"Chad","zone":"Africa/Ndjamena"},"tf":{"kmap":"","mirror":"","name":"French Southern Territories","zone":"Indian/Kerguelen"},"tg":{"kmap":"","mirror":"","name":"Togo","zone":"Africa/Lome"},"th":{"kmap":"","mirror":"","name":"Thailand","zone":"Asia/Bangkok"},"tj":{"kmap":"","mirror":"","name":"Tajikistan","zone":"Asia/Dushanbe"},"tk":{"kmap":"","mirror":"","name":"Tokelau","zone":"Pacific/Fakaofo"},"tl":{"kmap":"","mirror":"","name":"Timor-Leste","zone":"Asia/Dili"},"tm":{"kmap":"","mirror":"","name":"Turkmenistan","zone":"Asia/Ashgabat"},"tn":{"kmap":"","mirror":"","name":"Tunisia","zone":"Africa/Tunis"},"to":{"kmap":"","mirror":"","name":"Tonga","zone":"Pacific/Tongatapu"},"tr":{"kmap":"","mirror":"ftp.tr.debian.org","name":"Türkiye","zone":"Europe/Istanbul"},"tt":{"kmap":"","mirror":"","name":"Trinidad and Tobago","zone":"America/Port_of_Spain"},"tv":{"kmap":"","mirror":"","name":"Tuvalu","zone":"Pacific/Funafuti"},"tw":{"kmap":"","mirror":"ftp.tw.debian.org","name":"Taiwan","zone":"Asia/Taipei"},"tz":{"kmap":"","mirror":"","name":"Tanzania","zone":"Africa/Dar_es_Salaam"},"ua":{"kmap":"","mirror":"","name":"Ukraine","zone":"Europe/Simferopol"},"ug":{"kmap":"","mirror":"","name":"Uganda","zone":"Africa/Kampala"},"um":{"kmap":"","mirror":"","name":"United States Minor Outlying Islands","zone":"Pacific/Midway"},"us":{"kmap":"en-us","mirror":"ftp.us.debian.org","name":"United States","zone":"America/New_York"},"uy":{"kmap":"","mirror":"","name":"Uruguay","zone":"America/Montevideo"},"uz":{"kmap":"","mirror":"","name":"Uzbekistan","zone":"Asia/Samarkand"},"va":{"kmap":"it","mirror":"","name":"Holy See (Vatican City State)","zone":"Europe/Vatican"},"vc":{"kmap":"","mirror":"","name":"Saint Vincent and the Grenadines","zone":"America/St_Vincent"},"ve":{"kmap":"","mirror":"","name":"Venezuela","zone":"America/Caracas"},"vg":{"kmap":"","mirror":"","name":"Virgin Islands, British","zone":"America/Tortola"},"vi":{"kmap":"","mirror":"","name":"Virgin Islands, U.S.","zone":"America/St_Thomas"},"vn":{"kmap":"","mirror":"","name":"Vietnam","zone":"Asia/Ho_Chi_Minh"},"vu":{"kmap":"","mirror":"","name":"Vanuatu","zone":"Pacific/Efate"},"wf":{"kmap":"","mirror":"","name":"Wallis and Futuna","zone":"Pacific/Wallis"},"ws":{"kmap":"","mirror":"","name":"Samoa","zone":"Pacific/Apia"},"ye":{"kmap":"","mirror":"","name":"Yemen","zone":"Asia/Aden"},"yt":{"kmap":"","mirror":"","name":"Mayotte","zone":"Indian/Mayotte"},"za":{"kmap":"","mirror":"","name":"South Africa","zone":"Africa/Johannesburg"},"zm":{"kmap":"","mirror":"","name":"Zambia","zone":"Africa/Lusaka"},"zw":{"kmap":"","mirror":"","name":"Zimbabwe","zone":"Africa/Harare"}},"countryhash":{"afghanistan":"af","albania":"al","algeria":"dz","american samoa":"as","andorra":"ad","angola":"ao","anguilla":"ai","antarctica":"aq","antigua and barbuda":"ag","argentina":"ar","armenia":"am","aruba":"aw","australia":"au","austria":"at","azerbaijan":"az","bahamas":"bs","bahrain":"bh","bangladesh":"bd","barbados":"bb","belarus":"by","belgium":"be","belize":"bz","benin":"bj","bermuda":"bm","bhutan":"bt","bolivia":"bo","bonaire, sint eustatius and saba":"bq","bosnia and herzegovina":"ba","botswana":"bw","bouvet island":"bv","brazil":"br","british indian ocean territory":"io","brunei darussalam":"bn","bulgaria":"bg","burkina faso":"bf","burundi":"bi","cabo verde":"cv","cambodia":"kh","cameroon":"cm","canada":"ca","cayman islands":"ky","central african republic":"cf","chad":"td","chile":"cl","china":"cn","christmas island":"cx","cocos (keeling) islands":"cc","colombia":"co","comoros":"km","congo":"cg","congo, the democratic republic of the":"cd","cook islands":"ck","costa rica":"cr","croatia":"hr","cuba":"cu","curaçao":"cw","cyprus":"cy","czechia":"cz","côte d'ivoire":"ci","denmark":"dk","djibouti":"dj","dominica":"dm","dominican republic":"do","ecuador":"ec","egypt":"eg","el salvador":"sv","equatorial guinea":"gq","eritrea":"er","estonia":"ee","eswatini":"sz","ethiopia":"et","falkland islands (malvinas)":"fk","faroe islands":"fo","fiji":"fj","finland":"fi","france":"fr","french guiana":"gf","french polynesia":"pf","french southern territories":"tf","gabon":"ga","gambia":"gm","georgia":"ge","germany":"de","ghana":"gh","gibraltar":"gi","greece":"gr","greenland":"gl","grenada":"gd","guadeloupe":"gp","guam":"gu","guatemala":"gt","guernsey":"gg","guinea":"gn","guinea-bissau":"gw","guyana":"gy","haiti":"ht","heard island and mcdonald islands":"hm","holy see (vatican city state)":"va","honduras":"hn","hong kong":"hk","hungary":"hu","iceland":"is","india":"in","indonesia":"id","iran":"ir","iraq":"iq","ireland":"ie","isle of man":"im","israel":"il","italy":"it","jamaica":"jm","japan":"jp","jersey":"je","jordan":"jo","kazakhstan":"kz","kenya":"ke","kiribati":"ki","kuwait":"kw","kyrgyzstan":"kg","laos":"la","latvia":"lv","lebanon":"lb","lesotho":"ls","liberia":"lr","libya":"ly","liechtenstein":"li","lithuania":"lt","luxembourg":"lu","macao":"mo","madagascar":"mg","malawi":"mw","malaysia":"my","maldives":"mv","mali":"ml","malta":"mt","marshall islands":"mh","martinique":"mq","mauritania":"mr","mauritius":"mu","mayotte":"yt","mexico":"mx","micronesia, federated states of":"fm","moldova":"md","monaco":"mc","mongolia":"mn","montenegro":"me","montserrat":"ms","morocco":"ma","mozambique":"mz","myanmar":"mm","namibia":"na","nauru":"nr","nepal":"np","netherlands":"nl","new caledonia":"nc","new zealand":"nz","nicaragua":"ni","niger":"ne","nigeria":"ng","niue":"nu","norfolk island":"nf","north korea":"kp","north macedonia":"mk","northern mariana islands":"mp","norway":"no","oman":"om","pakistan":"pk","palau":"pw","palestine, state of":"ps","panama":"pa","papua new guinea":"pg","paraguay":"py","peru":"pe","philippines":"ph","pitcairn":"pn","poland":"pl","portugal":"pt","puerto rico":"pr","qatar":"qa","romania":"ro","russian federation":"ru","rwanda":"rw","réunion":"re","saint barthélemy":"bl","saint helena, ascension and tristan da cunha":"sh","saint kitts and nevis":"kn","saint lucia":"lc","saint martin (french part)":"mf","saint pierre and miquelon":"pm","saint vincent and the grenadines":"vc","samoa":"ws","san marino":"sm","sao tome and principe":"st","saudi arabia":"sa","senegal":"sn","serbia":"rs","seychelles":"sc","sierra leone":"sl","singapore":"sg","sint maarten (dutch part)":"sx","slovakia":"sk","slovenia":"si","solomon islands":"sb","somalia":"so","south africa":"za","south georgia and the south sandwich islands":"gs","south korea":"kr","south sudan":"ss","spain":"es","sri lanka":"lk","sudan":"sd","suriname":"sr","svalbard and jan mayen":"sj","sweden":"se","switzerland":"ch","syria":"sy","taiwan":"tw","tajikistan":"tj","tanzania":"tz","thailand":"th","timor-leste":"tl","togo":"tg","tokelau":"tk","tonga":"to","trinidad and tobago":"tt","tunisia":"tn","turkmenistan":"tm","turks and caicos islands":"tc","tuvalu":"tv","türkiye":"tr","uganda":"ug","ukraine":"ua","united arab emirates":"ae","united kingdom":"gb","united states":"us","united states minor outlying islands":"um","uruguay":"uy","uzbekistan":"uz","vanuatu":"vu","venezuela":"ve","vietnam":"vn","virgin islands, british":"vg","virgin islands, u.s.":"vi","wallis and futuna":"wf","western sahara":"eh","yemen":"ye","zambia":"zm","zimbabwe":"zw","åland islands":"ax"},"kmap":{"de":{"console":"qwertz/de-latin1-nodeadkeys.kmap.gz","kvm":"de","name":"German","x11":"de","x11var":"nodeadkeys"},"de-ch":{"console":"qwertz/sg-latin1.kmap.gz","kvm":"de-ch","name":"Swiss-German","x11":"ch","x11var":"de_nodeadkeys"},"dk":{"console":"qwerty/dk-latin1.kmap.gz","kvm":"da","name":"Danish","x11":"dk","x11var":"nodeadkeys"},"en-gb":{"console":"qwerty/uk.kmap.gz","kvm":"en-gb","name":"United Kingdom","x11":"gb","x11var":""},"en-us":{"console":"qwerty/us-latin1.kmap.gz","kvm":"en-us","name":"U.S. English","x11":"us","x11var":""},"es":{"console":"qwerty/es.kmap.gz","kvm":"es","name":"Spanish","x11":"es","x11var":"nodeadkeys"},"fi":{"console":"qwerty/fi-latin1.kmap.gz","kvm":"fi","name":"Finnish","x11":"fi","x11var":"nodeadkeys"},"fr":{"console":"azerty/fr-latin1.kmap.gz","kvm":"fr","name":"French","x11":"fr","x11var":"nodeadkeys"},"fr-be":{"console":"azerty/be2-latin1.kmap.gz","kvm":"fr-be","name":"Belgium-French","x11":"be","x11var":"nodeadkeys"},"fr-ca":{"console":"qwerty/cf.kmap.gz","kvm":"fr-ca","name":"Canada-French","x11":"ca","x11var":"fr-legacy"},"fr-ch":{"console":"qwertz/fr_CH-latin1.kmap.gz","kvm":"fr-ch","name":"Swiss-French","x11":"ch","x11var":"fr_nodeadkeys"},"hu":{"console":"qwertz/hu.kmap.gz","kvm":"hu","name":"Hungarian","x11":"hu","x11var":""},"is":{"console":"qwerty/is-latin1.kmap.gz","kvm":"is","name":"Icelandic","x11":"is","x11var":"nodeadkeys"},"it":{"console":"qwerty/it2.kmap.gz","kvm":"it","name":"Italian","x11":"it","x11var":"nodeadkeys"},"jp":{"console":"qwerty/jp106.kmap.gz","kvm":"ja","name":"Japanese","x11":"jp","x11var":""},"lt":{"console":"qwerty/lt.kmap.gz","kvm":"lt","name":"Lithuanian","x11":"lt","x11var":"std"},"mk":{"console":"qwerty/mk.kmap.gz","kvm":"mk","name":"Macedonian","x11":"mk","x11var":"nodeadkeys"},"nl":{"console":"qwerty/nl.kmap.gz","kvm":"nl","name":"Dutch","x11":"nl","x11var":""},"no":{"console":"qwerty/no-latin1.kmap.gz","kvm":"no","name":"Norwegian","x11":"no","x11var":"nodeadkeys"},"pl":{"console":"qwerty/pl.kmap.gz","kvm":"pl","name":"Polish","x11":"pl","x11var":""},"pt":{"console":"qwerty/pt-latin1.kmap.gz","kvm":"pt","name":"Portuguese","x11":"pt","x11var":"nodeadkeys"},"pt-br":{"console":"qwerty/br-latin1.kmap.gz","kvm":"pt-br","name":"Brazil-Portuguese","x11":"br","x11var":"nodeadkeys"},"se":{"console":"qwerty/se-latin1.kmap.gz","kvm":"sv","name":"Swedish","x11":"se","x11var":"nodeadkeys"},"si":{"console":"qwertz/slovene.kmap.gz","kvm":"sl","name":"Slovenian","x11":"si","x11var":""},"tr":{"console":"qwerty/trq.kmap.gz","kvm":"tr","name":"Turkish","x11":"tr","x11var":""}},"kmaphash":{"Belgium-French":"fr-be","Brazil-Portuguese":"pt-br","Canada-French":"fr-ca","Danish":"dk","Dutch":"nl","Finnish":"fi","French":"fr","German":"de","Hungarian":"hu","Icelandic":"is","Italian":"it","Japanese":"jp","Lithuanian":"lt","Macedonian":"mk","Norwegian":"no","Polish":"pl","Portuguese":"pt","Slovenian":"si","Spanish":"es","Swedish":"se","Swiss-French":"fr-ch","Swiss-German":"de-ch","Turkish":"tr","U.S. English":"en-us","United Kingdom":"en-gb"},"zones":{"Africa/Abidjan":1,"Africa/Accra":1,"Africa/Addis_Ababa":1,"Africa/Algiers":1,"Africa/Asmara":1,"Africa/Bamako":1,"Africa/Bangui":1,"Africa/Banjul":1,"Africa/Bissau":1,"Africa/Blantyre":1,"Africa/Brazzaville":1,"Africa/Bujumbura":1,"Africa/Cairo":1,"Africa/Casablanca":1,"Africa/Ceuta":1,"Africa/Conakry":1,"Africa/Dakar":1,"Africa/Dar_es_Salaam":1,"Africa/Djibouti":1,"Africa/Douala":1,"Africa/El_Aaiun":1,"Africa/Freetown":1,"Africa/Gaborone":1,"Africa/Harare":1,"Africa/Johannesburg":1,"Africa/Juba":1,"Africa/Kampala":1,"Africa/Khartoum":1,"Africa/Kigali":1,"Africa/Kinshasa":1,"Africa/Lagos":1,"Africa/Libreville":1,"Africa/Lome":1,"Africa/Luanda":1,"Africa/Lubumbashi":1,"Africa/Lusaka":1,"Africa/Malabo":1,"Africa/Maputo":1,"Africa/Maseru":1,"Africa/Mbabane":1,"Africa/Mogadishu":1,"Africa/Monrovia":1,"Africa/Nairobi":1,"Africa/Ndjamena":1,"Africa/Niamey":1,"Africa/Nouakchott":1,"Africa/Ouagadougou":1,"Africa/Porto-Novo":1,"Africa/Sao_Tome":1,"Africa/Tripoli":1,"Africa/Tunis":1,"Africa/Windhoek":1,"America/Adak":1,"America/Anchorage":1,"America/Anguilla":1,"America/Antigua":1,"America/Araguaina":1,"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1,"America/Aruba":1,"America/Asuncion":1,"America/Atikokan":1,"America/Bahia":1,"America/Bahia_Banderas":1,"America/Barbados":1,"America/Belem":1,"America/Belize":1,"America/Blanc-Sablon":1,"America/Boa_Vista":1,"America/Bogota":1,"America/Boise":1,"America/Cambridge_Bay":1,"America/Campo_Grande":1,"America/Cancun":1,"America/Caracas":1,"America/Cayenne":1,"America/Cayman":1,"America/Chicago":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Costa_Rica":1,"America/Creston":1,"America/Cuiaba":1,"America/Curacao":1,"America/Danmarkshavn":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Denver":1,"America/Detroit":1,"America/Dominica":1,"America/Edmonton":1,"America/Eirunepe":1,"America/El_Salvador":1,"America/Fort_Nelson":1,"America/Fortaleza":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Grand_Turk":1,"America/Grenada":1,"America/Guadeloupe":1,"America/Guatemala":1,"America/Guayaquil":1,"America/Guyana":1,"America/Halifax":1,"America/Havana":1,"America/Hermosillo":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Jamaica":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Kralendijk":1,"America/La_Paz":1,"America/Lima":1,"America/Los_Angeles":1,"America/Lower_Princes":1,"America/Maceio":1,"America/Managua":1,"America/Manaus":1,"America/Marigot":1,"America/Martinique":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Menominee":1,"America/Merida":1,"America/Metlakatla":1,"America/Mexico_City":1,"America/Miquelon":1,"America/Moncton":1,"America/Monterrey":1,"America/Montevideo":1,"America/Montserrat":1,"America/Nassau":1,"America/New_York":1,"America/Nome":1,"America/Noronha":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Nuuk":1,"America/Ojinaga":1,"America/Panama":1,"America/Paramaribo":1,"America/Phoenix":1,"America/Port-au-Prince":1,"America/Port_of_Spain":1,"America/Porto_Velho":1,"America/Puerto_Rico":1,"America/Punta_Arenas":1,"America/Rankin_Inlet":1,"America/Recife":1,"America/Regina":1,"America/Resolute":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Santiago":1,"America/Santo_Domingo":1,"America/Sao_Paulo":1,"America/Scoresbysund":1,"America/Sitka":1,"America/St_Barthelemy":1,"America/St_Johns":1,"America/St_Kitts":1,"America/St_Lucia":1,"America/St_Thomas":1,"America/St_Vincent":1,"America/Swift_Current":1,"America/Tegucigalpa":1,"America/Thule":1,"America/Tijuana":1,"America/Toronto":1,"America/Tortola":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1,"America/Yakutat":1,"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Macquarie":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1,"Arctic/Longyearbyen":1,"Asia/Aden":1,"Asia/Almaty":1,"Asia/Amman":1,"Asia/Anadyr":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Ashgabat":1,"Asia/Atyrau":1,"Asia/Baghdad":1,"Asia/Bahrain":1,"Asia/Baku":1,"Asia/Bangkok":1,"Asia/Barnaul":1,"Asia/Beirut":1,"Asia/Bishkek":1,"Asia/Brunei":1,"Asia/Chita":1,"Asia/Choibalsan":1,"Asia/Colombo":1,"Asia/Damascus":1,"Asia/Dhaka":1,"Asia/Dili":1,"Asia/Dubai":1,"Asia/Dushanbe":1,"Asia/Famagusta":1,"Asia/Gaza":1,"Asia/Hebron":1,"Asia/Ho_Chi_Minh":1,"Asia/Hong_Kong":1,"Asia/Hovd":1,"Asia/Irkutsk":1,"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Jerusalem":1,"Asia/Kabul":1,"Asia/Kamchatka":1,"Asia/Karachi":1,"Asia/Kathmandu":1,"Asia/Khandyga":1,"Asia/Kolkata":1,"Asia/Krasnoyarsk":1,"Asia/Kuala_Lumpur":1,"Asia/Kuching":1,"Asia/Kuwait":1,"Asia/Macau":1,"Asia/Magadan":1,"Asia/Makassar":1,"Asia/Manila":1,"Asia/Muscat":1,"Asia/Nicosia":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Oral":1,"Asia/Phnom_Penh":1,"Asia/Pontianak":1,"Asia/Pyongyang":1,"Asia/Qatar":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1,"Asia/Riyadh":1,"Asia/Sakhalin":1,"Asia/Samarkand":1,"Asia/Seoul":1,"Asia/Shanghai":1,"Asia/Singapore":1,"Asia/Srednekolymsk":1,"Asia/Taipei":1,"Asia/Tashkent":1,"Asia/Tbilisi":1,"Asia/Tehran":1,"Asia/Thimphu":1,"Asia/Tokyo":1,"Asia/Tomsk":1,"Asia/Ulaanbaatar":1,"Asia/Urumqi":1,"Asia/Ust-Nera":1,"Asia/Vientiane":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yangon":1,"Asia/Yekaterinburg":1,"Asia/Yerevan":1,"Atlantic/Azores":1,"Atlantic/Bermuda":1,"Atlantic/Canary":1,"Atlantic/Cape_Verde":1,"Atlantic/Faroe":1,"Atlantic/Madeira":1,"Atlantic/Reykjavik":1,"Atlantic/South_Georgia":1,"Atlantic/St_Helena":1,"Atlantic/Stanley":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1,"Europe/Amsterdam":1,"Europe/Andorra":1,"Europe/Astrakhan":1,"Europe/Athens":1,"Europe/Belgrade":1,"Europe/Berlin":1,"Europe/Bratislava":1,"Europe/Brussels":1,"Europe/Bucharest":1,"Europe/Budapest":1,"Europe/Busingen":1,"Europe/Chisinau":1,"Europe/Copenhagen":1,"Europe/Dublin":1,"Europe/Gibraltar":1,"Europe/Guernsey":1,"Europe/Helsinki":1,"Europe/Isle_of_Man":1,"Europe/Istanbul":1,"Europe/Jersey":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Kyiv":1,"Europe/Lisbon":1,"Europe/Ljubljana":1,"Europe/London":1,"Europe/Luxembourg":1,"Europe/Madrid":1,"Europe/Malta":1,"Europe/Mariehamn":1,"Europe/Minsk":1,"Europe/Monaco":1,"Europe/Moscow":1,"Europe/Oslo":1,"Europe/Paris":1,"Europe/Podgorica":1,"Europe/Prague":1,"Europe/Riga":1,"Europe/Rome":1,"Europe/Samara":1,"Europe/San_Marino":1,"Europe/Sarajevo":1,"Europe/Saratov":1,"Europe/Simferopol":1,"Europe/Skopje":1,"Europe/Sofia":1,"Europe/Stockholm":1,"Europe/Tallinn":1,"Europe/Tirane":1,"Europe/Ulyanovsk":1,"Europe/Vaduz":1,"Europe/Vatican":1,"Europe/Vienna":1,"Europe/Vilnius":1,"Europe/Volgograd":1,"Europe/Warsaw":1,"Europe/Zagreb":1,"Europe/Zurich":1,"Indian/Antananarivo":1,"Indian/Chagos":1,"Indian/Christmas":1,"Indian/Cocos":1,"Indian/Comoro":1,"Indian/Kerguelen":1,"Indian/Mahe":1,"Indian/Maldives":1,"Indian/Mauritius":1,"Indian/Mayotte":1,"Indian/Reunion":1,"Pacific/Apia":1,"Pacific/Auckland":1,"Pacific/Bougainville":1,"Pacific/Chatham":1,"Pacific/Chuuk":1,"Pacific/Easter":1,"Pacific/Efate":1,"Pacific/Fakaofo":1,"Pacific/Fiji":1,"Pacific/Funafuti":1,"Pacific/Galapagos":1,"Pacific/Gambier":1,"Pacific/Guadalcanal":1,"Pacific/Guam":1,"Pacific/Honolulu":1,"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Kosrae":1,"Pacific/Kwajalein":1,"Pacific/Majuro":1,"Pacific/Marquesas":1,"Pacific/Midway":1,"Pacific/Nauru":1,"Pacific/Niue":1,"Pacific/Norfolk":1,"Pacific/Noumea":1,"Pacific/Pago_Pago":1,"Pacific/Palau":1,"Pacific/Pitcairn":1,"Pacific/Pohnpei":1,"Pacific/Port_Moresby":1,"Pacific/Rarotonga":1,"Pacific/Saipan":1,"Pacific/Tahiti":1,"Pacific/Tarawa":1,"Pacific/Tongatapu":1,"Pacific/Wake":1,"Pacific/Wallis":1}} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json new file mode 100644 index 0000000..f228b1e --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json @@ -0,0 +1,29 @@ +{ + "autoreboot": 1, + "cidr": "192.168.1.114/24", + "country": "at", + "dns": "192.168.1.254", + "domain": "testinstall", + "disk_selection": { + "6": "6", + "7": "7", + "8": "8", + "9": "9" + }, + "filesys": "zfs (RAID10)", + "gateway": "192.168.1.1", + "hdsize": 223.57088470458984, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "eno1", + "password": "123456", + "timezone": "Europe/Vienna", + "zfs_opts": { + "arc_max": 2048, + "ashift": 12, + "checksum": "on", + "compress": "on", + "copies": 1 + } +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml new file mode 100644 index 0000000..796ccc6 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml @@ -0,0 +1,14 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = true + +[disks] +filesystem = "zfs-raid10" +filter.ID_SERIAL = "*MZ7KM240HAGR*" diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json new file mode 100644 index 0000000..2990922 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json @@ -0,0 +1,26 @@ +{ + "autoreboot": 1, + "cidr": "192.168.1.114/24", + "country": "at", + "dns": "192.168.1.254", + "domain": "testinstall", + "disk_selection": { + "9": "9" + }, + "filesys": "zfs (RAID0)", + "gateway": "192.168.1.1", + "hdsize": 223.57088470458984, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "eno1", + "password": "123456", + "timezone": "Europe/Vienna", + "zfs_opts": { + "arc_max": 2048, + "ashift": 12, + "checksum": "on", + "compress": "on", + "copies": 1 + } +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml new file mode 100644 index 0000000..5171153 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml @@ -0,0 +1,16 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = true + +[disks] +filesystem = "zfs-raid0" +filter_match = "all" +filter.ID_SERIAL = "*MZ7KM240HAGR*" +filter.ID_SERIAL_SHORT = "S2HRNX0J403419" diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json new file mode 100644 index 0000000..8c22300 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json @@ -0,0 +1,33 @@ +{ + "autoreboot": 1, + "cidr": "192.168.1.114/24", + "country": "at", + "dns": "192.168.1.254", + "domain": "testinstall", + "disk_selection": { + "0": "0", + "1": "1", + "2": "2", + "3": "3", + "6": "6", + "7": "7", + "8": "8", + "9": "9" + }, + "filesys": "zfs (RAID10)", + "gateway": "192.168.1.1", + "hdsize": 2980.820640563965, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "eno1", + "password": "123456", + "timezone": "Europe/Vienna", + "zfs_opts": { + "arc_max": 2048, + "ashift": 12, + "checksum": "on", + "compress": "on", + "copies": 1 + } +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml new file mode 100644 index 0000000..df6c88c --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml @@ -0,0 +1,16 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = true + +[disks] +filesystem = "zfs-raid10" +filter_match = "any" +filter.ID_SERIAL = "*MZ7KM240HAGR*" +filter.ID_MODEL = "Micron_9300*" diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.json b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json new file mode 100644 index 0000000..9021377 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json @@ -0,0 +1,17 @@ +{ + "autoreboot": 1, + "cidr": "192.168.1.114/24", + "country": "at", + "dns": "192.168.1.254", + "domain": "testinstall", + "filesys": "ext4", + "gateway": "192.168.1.1", + "hdsize": 223.57088470458984, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "eno1", + "password": "123456", + "target_hd": "/dev/sda", + "timezone": "Europe/Vienna" +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml new file mode 100644 index 0000000..a417c00 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml @@ -0,0 +1,14 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = true + +[disks] +filesystem = "ext4" +disk_selection = ["sda"] diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json new file mode 100644 index 0000000..6f31079 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json @@ -0,0 +1,17 @@ +{ + "autoreboot": 1, + "cidr": "10.10.10.10/24", + "country": "at", + "dns": "10.10.10.1", + "domain": "testinstall", + "filesys": "ext4", + "gateway": "10.10.10.1", + "hdsize": 223.57088470458984, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "enp65s0f0", + "password": "123456", + "target_hd": "/dev/sda", + "timezone": "Europe/Vienna" +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml new file mode 100644 index 0000000..10e21d2 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml @@ -0,0 +1,19 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = false +cidr = "10.10.10.10/24" +dns = "10.10.10.1" +gateway = "10.10.10.1" +filter.ID_NET_NAME_MAC = "*a0369f0ab382" + + +[disks] +filesystem = "ext4" +disk_selection = ["sda"] diff --git a/proxmox-auto-installer/tests/resources/parse_answer/readme b/proxmox-auto-installer/tests/resources/parse_answer/readme new file mode 100644 index 0000000..6ce77ae --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/readme @@ -0,0 +1,4 @@ +the size parameter from /sys/block/{disk}/size is the number of blocks. + +to calculate the size as the low level installer needs it: +size * 512 / 1024 / 1024 / 1024 with 14 digits after the comma diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json new file mode 100644 index 0000000..515cc89 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json @@ -0,0 +1,17 @@ +{ + "autoreboot": 1, + "cidr": "10.10.10.10/24", + "country": "at", + "dns": "10.10.10.1", + "domain": "testinstall", + "filesys": "ext4", + "gateway": "10.10.10.1", + "hdsize": 223.57088470458984, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "enp129s0f1np1", + "password": "123456", + "target_hd": "/dev/sda", + "timezone": "Europe/Vienna" +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml new file mode 100644 index 0000000..60c08b4 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml @@ -0,0 +1,19 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = false +cidr = "10.10.10.10/24" +dns = "10.10.10.1" +gateway = "10.10.10.1" +filter.ID_NET_NAME = "enp129s0f1np1" + + +[disks] +filesystem = "ext4" +disk_selection = ["sda"] diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.json b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json new file mode 100644 index 0000000..c29a303 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json @@ -0,0 +1,27 @@ +{ + "autoreboot": 1, + "cidr": "192.168.1.114/24", + "country": "at", + "dns": "192.168.1.254", + "domain": "testinstall", + "disk_selection": { + "6": "6", + "7": "7" + }, + "filesys": "zfs (RAID1)", + "gateway": "192.168.1.1", + "hdsize": 80.0, + "hostname": "pveauto", + "keymap": "de", + "mailto": "mail@no.invalid", + "mngmt_nic": "eno1", + "password": "123456", + "timezone": "Europe/Vienna", + "zfs_opts": { + "arc_max": 2048, + "ashift": 12, + "checksum": "on", + "compress": "lz4", + "copies": 2 + } +} diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml new file mode 100644 index 0000000..2032c0a --- /dev/null +++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml @@ -0,0 +1,19 @@ +[global] +keyboard = "de" +country = "at" +fqdn = "pveauto.testinstall" +mailto = "mail@no.invalid" +timezone = "Europe/Vienna" +password = "123456" + +[network] +use_dhcp = true + +[disks] +filesystem = "zfs-raid1" +disk_selection = ["sda", "sdb"] +zfs.ashift = 12 +zfs.checksum = "on" +zfs.compress = "lz4" +zfs.copies = 2 +zfs.hdsize = 80 diff --git a/proxmox-auto-installer/tests/resources/run-env-info.json b/proxmox-auto-installer/tests/resources/run-env-info.json new file mode 100644 index 0000000..6762470 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/run-env-info.json @@ -0,0 +1 @@ +{"boot_type":"efi","country":"at","disks":[[0,"/dev/nvme0n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme0n1"],[1,"/dev/nvme1n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme1n1"],[2,"/dev/nvme2n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme2n1"],[3,"/dev/nvme3n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme3n1"],[4,"/dev/nvme4n1",976773168,"Samsung SSD 970 EVO Plus 500GB",512,"/sys/block/nvme4n1"],[5,"/dev/nvme5n1",732585168,"INTEL SSDPED1K375GA",512,"/sys/block/nvme5n1"],[6,"/dev/sda",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sda"],[7,"/dev/sdb",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdb"],[8,"/dev/sdc",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdc"],[9,"/dev/sdd",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdd"]],"hvm_supported":1,"ipconf":{"default":"4","dnsserver":"192.168.1.254","domain":null,"gateway":"192.168.1.1","ifaces":{"10":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"2":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"3":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"4":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","inet":{"addr":"192.168.1.114","mask":"255.255.240.0","prefix":20},"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"5":{"driver":"cdc_ether","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"},"6":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"7":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"8":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"9":{"driver":"mlx5_core","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"}}},"kernel_cmdline":"BOOT_IMAGE=/boot/linux26 ro ramdisk_size=16777216 rw splash=verbose proxdebug vga=788","network":{"dns":{"dns":["192.168.1.254"],"domain":null},"interfaces":{"eno1":{"addresses":[{"address":"192.168.1.114","family":"inet","prefix":24}],"index":4,"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"eno2":{"index":6,"mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"enp129s0f0np0":{"index":7,"mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"enp129s0f1np1":{"index":8,"mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"enp193s0f0np0":{"index":9,"mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"},"enp193s0f1np1":{"index":10,"mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"enp65s0f0":{"index":2,"mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"enp65s0f1":{"index":3,"mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"enx5a4732ddc747":{"index":5,"mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"}},"routes":{"gateway4":{"dev":"eno1","gateway":"192.168.1.1"}}},"total_memory":257597} diff --git a/proxmox-auto-installer/tests/resources/run-env-udev.json b/proxmox-auto-installer/tests/resources/run-env-udev.json new file mode 100644 index 0000000..4fe1f30 --- /dev/null +++ b/proxmox-auto-installer/tests/resources/run-env-udev.json @@ -0,0 +1 @@ +{"disks":{"0":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:01:00.0-nvme-1 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_19502596FC74 /dev/disk/by-id/lvm-pv-uuid-hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP /dev/disk/by-diskseq/16 /dev/disk/by-id/nvme-eui.000000000000001500a075012596fc74","DEVNAME":"/dev/nvme0n1","DEVPATH":"/devices/pci0000:00/0000:00:01.1/0000:01:00.0/nvme/nvme0/nvme0n1","DEVTYPE":"disk","DISKSEQ":"16","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_UUID_ENC":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:01:00.0-nvme-1","ID_PATH_TAG":"pci-0000_01_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_19502596FC74","ID_SERIAL_SHORT":"19502596FC74","ID_WWN":"eui.000000000000001500a075012596fc74","LVM_VG_NAME_COMPLETE":"ceph-67f6a633-8bac-4ba6-a54c-40f0d24a9701","MAJOR":"259","MINOR":"6","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215609"},"1":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:02:00.0-nvme-1 /dev/disk/by-id/nvme-eui.000000000000001400a0750125de7a16 /dev/disk/by-diskseq/15 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_195225DE7A16","DEVNAME":"/dev/nvme1n1","DEVPATH":"/devices/pci0000:00/0000:00:01.2/0000:02:00.0/nvme/nvme1/nvme1n1","DEVTYPE":"disk","DISKSEQ":"15","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:02:00.0-nvme-1","ID_PATH_TAG":"pci-0000_02_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_195225DE7A16","ID_SERIAL_SHORT":"195225DE7A16","ID_WWN":"eui.000000000000001400a0750125de7a16","MAJOR":"259","MINOR":"5","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271971"},"2":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:03:00.0-nvme-1 /dev/disk/by-diskseq/17 /dev/disk/by-id/lvm-pv-uuid-b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F206E /dev/disk/by-id/nvme-eui.000000000000001400a07501250f206e","DEVNAME":"/dev/nvme2n1","DEVPATH":"/devices/pci0000:00/0000:00:01.3/0000:03:00.0/nvme/nvme2/nvme2n1","DEVTYPE":"disk","DISKSEQ":"17","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_UUID_ENC":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:03:00.0-nvme-1","ID_PATH_TAG":"pci-0000_03_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F206E","ID_SERIAL_SHORT":"1945250F206E","ID_WWN":"eui.000000000000001400a07501250f206e","LVM_VG_NAME_COMPLETE":"ceph-ee820014-6121-458b-a661-889f0901bff6","MAJOR":"259","MINOR":"7","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45218640"},"3":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:04:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F20AC /dev/disk/by-diskseq/18 /dev/disk/by-id/nvme-eui.000000000000001400a07501250f20ac","DEVNAME":"/dev/nvme3n1","DEVPATH":"/devices/pci0000:00/0000:00:01.4/0000:04:00.0/nvme/nvme3/nvme3n1","DEVTYPE":"disk","DISKSEQ":"18","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_UUID_ENC":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:04:00.0-nvme-1","ID_PATH_TAG":"pci-0000_04_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F20AC","ID_SERIAL_SHORT":"1945250F20AC","ID_WWN":"eui.000000000000001400a07501250f20ac","LVM_VG_NAME_COMPLETE":"ceph-2928aceb-9300-4175-8640-e227d897d45e","MAJOR":"259","MINOR":"8","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215244"},"4":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/13 /dev/disk/by-path/pci-0000:82:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd /dev/disk/by-id/nvme-INTEL_SSDPED1K375GA_PHKS746500DK375AGN /dev/disk/by-id/nvme-nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","DEVNAME":"/dev/nvme4n1","DEVPATH":"/devices/pci0000:80/0000:80:03.1/0000:82:00.0/nvme/nvme4/nvme4n1","DEVTYPE":"disk","DISKSEQ":"13","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_UUID_ENC":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_VERSION":"LVM2 001","ID_MODEL":"INTEL SSDPED1K375GA","ID_PATH":"pci-0000:82:00.0-nvme-1","ID_PATH_TAG":"pci-0000_82_00_0-nvme-1","ID_REVISION":"E2010435","ID_SERIAL":"INTEL_SSDPED1K375GA_PHKS746500DK375AGN","ID_SERIAL_SHORT":"PHKS746500DK375AGN","ID_WWN":"nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","LVM_VG_NAME_COMPLETE":"ceph-b4af8112-88e7-4cd4-9cf9-0f4163ca77bd","MAJOR":"259","MINOR":"0","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45219471"},"5":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/nvme-eui.0025385791b04175 /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N /dev/disk/by-path/pci-0000:06:00.0-nvme-1 /dev/disk/by-diskseq/14","DEVNAME":"/dev/nvme5n1","DEVPATH":"/devices/pci0000:00/0000:00:03.3/0000:06:00.0/nvme/nvme5/nvme5n1","DEVTYPE":"disk","DISKSEQ":"14","ID_MODEL":"Samsung SSD 970 EVO Plus 500GB","ID_PART_TABLE_TYPE":"gpt","ID_PART_TABLE_UUID":"1c40cb4b-72d8-49ec-804b-e5933e09423d","ID_PATH":"pci-0000:06:00.0-nvme-1","ID_PATH_TAG":"pci-0000_06_00_0-nvme-1","ID_REVISION":"2B2QEXM7","ID_SERIAL":"Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N","ID_SERIAL_SHORT":"S4EVNF0M703256N","ID_WWN":"eui.0025385791b04175","MAJOR":"259","MINOR":"1","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271933"},"6":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/lvm-pv-uuid-tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0 /dev/disk/by-diskseq/9 /dev/disk/by-id/wwn-0x5002538c405dbf10","DEVNAME":"/dev/sda","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:0/end_device-8:0:0/target8:0:0/8:0:0:0/block/sda","DEVTYPE":"disk","DISKSEQ":"9","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_UUID_ENC":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550","ID_SERIAL_SHORT":"S2HRNX0J403550","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbf10","ID_WWN_WITH_EXTENSION":"0x5002538c405dbf10","MAJOR":"8","MINOR":"0","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45234812"},"7":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbce5 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0 /dev/disk/by-id/lvm-pv-uuid-oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu /dev/disk/by-diskseq/10 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","DEVNAME":"/dev/sdb","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:1/end_device-8:0:1/target8:0:1/8:0:1:0/block/sdb","DEVTYPE":"disk","DISKSEQ":"10","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_UUID_ENC":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","ID_SERIAL_SHORT":"S2HRNX0J403335","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbce5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbce5","MAJOR":"8","MINOR":"16","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215406"},"8":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbcd9 /dev/disk/by-diskseq/11 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0 /dev/disk/by-id/lvm-pv-uuid-tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","DEVNAME":"/dev/sdc","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:2/end_device-8:0:2/target8:0:2/8:0:2:0/block/sdc","DEVTYPE":"disk","DISKSEQ":"11","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_UUID_ENC":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333","ID_SERIAL_SHORT":"S2HRNX0J403333","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbcd9","ID_WWN_WITH_EXTENSION":"0x5002538c405dbcd9","MAJOR":"8","MINOR":"32","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45198824"},"9":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/12 /dev/disk/by-id/wwn-0x5002538c405dbdc5 /dev/disk/by-id/lvm-pv-uuid-Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","DEVNAME":"/dev/sdd","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:3/end_device-8:0:3/target8:0:3/8:0:3:0/block/sdd","DEVTYPE":"disk","DISKSEQ":"12","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_UUID_ENC":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","ID_SERIAL_SHORT":"S2HRNX0J403419","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbdc5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbdc5","MAJOR":"8","MINOR":"48","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215283"}},"nics":{"eno1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.0/net/eno1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN1","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno1","ID_NET_NAME_MAC":"enxb42e99acadb4","ID_NET_NAME_ONBOARD":"eno1","ID_NET_NAME_PATH":"enp194s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.0","ID_PATH_TAG":"pci-0000_c2_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"5","INTERFACE":"eno1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno1","TAGS":":systemd:","USEC_INITIALIZED":"45212091"},"eno2":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.1/net/eno2","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN2","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno2","ID_NET_NAME_MAC":"enxb42e99acadb5","ID_NET_NAME_ONBOARD":"eno2","ID_NET_NAME_PATH":"enp194s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.1","ID_PATH_TAG":"pci-0000_c2_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"6","INTERFACE":"eno2","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno2","TAGS":":systemd:","USEC_INITIALIZED":"45128159"},"enp129s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.0/net/enp129s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f0np0","ID_NET_NAME_MAC":"enx1c34da5c5e24","ID_NET_NAME_PATH":"enp129s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.0","ID_PATH_TAG":"pci-0000_81_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"7","INTERFACE":"enp129s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47752091"},"enp129s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.1/net/enp129s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f1np1","ID_NET_NAME_MAC":"enx1c34da5c5e25","ID_NET_NAME_PATH":"enp129s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.1","ID_PATH_TAG":"pci-0000_81_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"8","INTERFACE":"enp129s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47716100"},"enp193s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.0/net/enp193s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f0np0","ID_NET_NAME_MAC":"enx248a071e05bc","ID_NET_NAME_PATH":"enp193s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.0","ID_PATH_TAG":"pci-0000_c1_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"9","INTERFACE":"enp193s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47784094"},"enp193s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.1/net/enp193s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f1np1","ID_NET_NAME_MAC":"enx248a071e05bd","ID_NET_NAME_PATH":"enp193s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.1","ID_PATH_TAG":"pci-0000_c1_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"10","INTERFACE":"enp193s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47820155"},"enp65s0f0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.0/net/enp65s0f0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f0","ID_NET_NAME_MAC":"enxa0369f0ab382","ID_NET_NAME_PATH":"enp65s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.0","ID_PATH_TAG":"pci-0000_41_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"3","INTERFACE":"enp65s0f0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f0","TAGS":":systemd:","USEC_INITIALIZED":"45176103"},"enp65s0f1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.1/net/enp65s0f1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f1","ID_NET_NAME_MAC":"enxa0369f0ab383","ID_NET_NAME_PATH":"enp65s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.1","ID_PATH_TAG":"pci-0000_41_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"4","INTERFACE":"enp65s0f1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f1","TAGS":":systemd:","USEC_INITIALIZED":"45260218"},"enxaa0c304b6362":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:08.1/0000:43:00.3/usb3/3-2/3-2.4/3-2.4.3/3-2.4.3:2.0/net/enxaa0c304b6362","ID_BUS":"usb","ID_MODEL":"Virtual_Ethernet","ID_MODEL_ENC":"Virtual\\x20Ethernet","ID_MODEL_ID":"ffb0","ID_NET_DRIVER":"cdc_ether","ID_NET_LINK_FILE":"/usr/lib/systemd/network/73-usb-net-by-mac.link","ID_NET_NAME":"enxaa0c304b6362","ID_NET_NAME_MAC":"enxaa0c304b6362","ID_NET_NAME_PATH":"enp67s0f3u2u4u3c2","ID_NET_NAMING_SCHEME":"v252","ID_PATH":"pci-0000:43:00.3-usb-0:2.4.3:2.0","ID_PATH_TAG":"pci-0000_43_00_3-usb-0_2_4_3_2_0","ID_REVISION":"0100","ID_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_SERIAL_SHORT":"1234567890","ID_TYPE":"generic","ID_USB_CLASS_FROM_DATABASE":"Communications","ID_USB_DRIVER":"cdc_ether","ID_USB_INTERFACES":":0202ff:0a0000:020600:","ID_USB_INTERFACE_NUM":"00","ID_USB_MODEL":"Virtual_Ethernet","ID_USB_MODEL_ENC":"Virtual\\x20Ethernet","ID_USB_MODEL_ID":"ffb0","ID_USB_REVISION":"0100","ID_USB_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_USB_SERIAL_SHORT":"1234567890","ID_USB_TYPE":"generic","ID_USB_VENDOR":"American_Megatrends_Inc.","ID_USB_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_USB_VENDOR_ID":"046b","ID_VENDOR":"American_Megatrends_Inc.","ID_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_VENDOR_FROM_DATABASE":"American Megatrends, Inc.","ID_VENDOR_ID":"046b","IFINDEX":"2","INTERFACE":"enxaa0c304b6362","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enxaa0c304b6362","TAGS":":systemd:","USEC_INITIALIZED":"44748106"}}} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 12/22] auto-installer: add auto-installer binary 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (10 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 11/22] auto-installer: add tests for answer file parsing Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 13/22] auto-installer: add fetch answer binary Aaron Lauterer ` (11 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel It expects the contents of an answer file via stdin. It will then be parsed and the JSON for the low level installer is generated. It then calls the low level installer directly. The output of the installaton progress is kept rather simple for now. If configured in the answer file, commands will be run pre and post the low level installer. It also logs everything to the logfile, currently '/tmp/auto_installer.log'. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- Makefile | 4 +- .../src/bin/proxmox-auto-installer.rs | 193 ++++++++++++++++++ 2 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs diff --git a/Makefile b/Makefile index 57bd7ae..9c933d9 100644 --- a/Makefile +++ b/Makefile @@ -18,7 +18,9 @@ INSTALLER_SOURCES=$(shell git ls-files) country.dat PREFIX = /usr BINDIR = $(PREFIX)/bin -USR_BIN := proxmox-tui-installer +USR_BIN := \ + proxmox-tui-installer\ + proxmox-auto-installer COMPILED_BINS := \ $(addprefix $(CARGO_COMPILEDIR)/,$(USR_BIN)) diff --git a/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs new file mode 100644 index 0000000..ea8aa00 --- /dev/null +++ b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs @@ -0,0 +1,193 @@ +use anyhow::{anyhow, bail, Error, Result}; +use log::{error, info, LevelFilter}; +use std::{ + env, + io::{BufRead, BufReader, Write}, + path::PathBuf, + process::ExitCode, +}; + +use proxmox_installer_common::setup::{ + installer_setup, read_json, spawn_low_level_installer, LocaleInfo, RuntimeInfo, SetupInfo, +}; + +use proxmox_auto_installer::{ + answer::Answer, + log::AutoInstLogger, + udevinfo::UdevInfo, + utils, + utils::{parse_answer, LowLevelMessage}, +}; + +static LOGGER: AutoInstLogger = AutoInstLogger; + +pub fn init_log() -> Result<()> { + AutoInstLogger::init("/tmp/auto_installer.log")?; + log::set_logger(&LOGGER) + .map(|()| log::set_max_level(LevelFilter::Info)) + .map_err(|err| anyhow!(err)) +} + +fn auto_installer_setup(in_test_mode: bool) -> Result<(Answer, UdevInfo)> { + let base_path = if in_test_mode { "./testdir" } else { "/" }; + let mut path = PathBuf::from(base_path); + + path.push("run"); + path.push("proxmox-installer"); + + let udev_info: UdevInfo = { + let mut path = path.clone(); + path.push("run-env-udev.json"); + + read_json(&path).map_err(|err| anyhow!("Failed to retrieve udev info details: {err}"))? + }; + + let mut buffer = String::new(); + let lines = std::io::stdin().lock().lines(); + for line in lines { + buffer.push_str(&line.unwrap()); + buffer.push('\n'); + } + + let answer: Answer = + toml::from_str(&buffer).map_err(|err| anyhow!("Failed parsing answer file: {err}"))?; + + Ok((answer, udev_info)) +} + +fn main() -> ExitCode { + if let Err(err) = init_log() { + panic!("could not initilize logging: {}", err); + } + + let in_test_mode = match env::args().nth(1).as_deref() { + Some("-t") => true, + // Always force the test directory in debug builds + _ => cfg!(debug_assertions), + }; + info!("Starting auto installer"); + + let (setup_info, locales, runtime_info) = match installer_setup(in_test_mode) { + Ok(result) => result, + Err(err) => { + error!("Installer setup error: {err}"); + return ExitCode::FAILURE; + } + }; + + let (answer, udevadm_info) = match auto_installer_setup(in_test_mode) { + Ok(result) => result, + Err(err) => { + error!("Autoinstaller setup error: {err}"); + return ExitCode::FAILURE; + } + }; + + match utils::run_cmds("Pre", &answer.global.pre_command) { + Ok(_) => (), + Err(err) => { + error!("Error when running Pre-Commands: {}", err); + return exit_failure(answer.global.reboot_on_error); + } + }; + match run_installation(&answer, &locales, &runtime_info, &udevadm_info, &setup_info) { + Ok(_) => info!("Installation done."), + Err(err) => { + error!("Installation failed: {err}"); + return exit_failure(answer.global.reboot_on_error); + } + } + match utils::run_cmds("Post", &answer.global.post_command) { + Ok(_) => (), + Err(err) => { + error!("Error when running Post-Commands: {}", err); + return exit_failure(answer.global.reboot_on_error); + } + }; + ExitCode::SUCCESS +} + +/// When we exit with a failure, the installer will not automatically reboot. +/// Default value for reboot_on_error is false +fn exit_failure(reboot_on_error: Option<bool>) -> ExitCode { + if let Some(true) = reboot_on_error { + ExitCode::SUCCESS + } else { + ExitCode::FAILURE + } +} + +fn run_installation( + answer: &Answer, + locales: &LocaleInfo, + runtime_info: &RuntimeInfo, + udevadm_info: &UdevInfo, + setup_info: &SetupInfo, +) -> Result<()> { + let config = parse_answer(answer, udevadm_info, runtime_info, locales, setup_info)?; + info!("Calling low-level installer"); + + let mut child = match spawn_low_level_installer(false) { + Ok(child) => child, + Err(err) => { + bail!("Low level installer could not be started: {}", err); + } + }; + + let mut cur_counter = 111; + let mut inner = || -> Result<()> { + let reader = child + .stdout + .take() + .map(BufReader::new) + .ok_or(anyhow!("failed to get stdout reader"))?; + let mut writer = child + .stdin + .take() + .ok_or(anyhow!("failed to get stdin writer"))?; + + serde_json::to_writer(&mut writer, &config) + .map_err(|err| anyhow!("failed to serialize install config: {err}"))?; + writeln!(writer).map_err(|err| anyhow!("failed to write install config: {err}"))?; + + for line in reader.lines() { + let line = match line { + Ok(line) => line, + Err(_) => break, + }; + let msg = match serde_json::from_str::<LowLevelMessage>(&line) { + Ok(msg) => msg, + Err(_) => { + // Not a fatal error, so don't abort the installation by returning + continue; + } + }; + + match msg.clone() { + LowLevelMessage::Info { message } => info!("{message}"), + LowLevelMessage::Error { message } => error!("{message}"), + LowLevelMessage::Prompt { query } => info!("Got Query: {query}"), + LowLevelMessage::Progress { ratio, text: _ } => { + let counter = (ratio * 100.).floor() as usize; + if counter != cur_counter { + cur_counter = counter; + info!("Progress: {counter:>3}%"); + } + } + LowLevelMessage::Finished { state, message } => { + if state == "err" { + bail!("{message}"); + } + info!("Finished: '{state}' {message}"); + } + }; + } + Ok(()) + }; + match inner() { + Err(err) => Err(Error::msg(format!( + "low level installer returned early: {err}" + ))), + _ => Ok(()), + } +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 13/22] auto-installer: add fetch answer binary 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (11 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 12/22] auto-installer: add auto-installer binary Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 14/22] unconfigured: add proxauto as option to start auto installer Aaron Lauterer ` (10 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel it is supposed to be run first and fetch an answer file. The initial implementation searches for a partition/filesystem called 'proxmoxinst' or 'PROXMOXINST' with an 'answer.toml' file in the root directory. Once it has an answer file, it will call the 'proxmox-auto-installer' and pipe in the contents via stdin. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- changes since v1: * removed hard coded paths to binaries * create mount dir with fs::create_dir_all Makefile | 1 + .../src/bin/proxmox-fetch-answer.rs | 71 +++++++++++++++ .../src/fetch_plugins/mod.rs | 2 + .../src/fetch_plugins/partition.rs | 32 +++++++ .../src/fetch_plugins/utils.rs | 90 +++++++++++++++++++ proxmox-auto-installer/src/lib.rs | 1 + 6 files changed, 197 insertions(+) create mode 100644 proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs create mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs create mode 100644 proxmox-auto-installer/src/fetch_plugins/partition.rs create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils.rs diff --git a/Makefile b/Makefile index 9c933d9..b724789 100644 --- a/Makefile +++ b/Makefile @@ -20,6 +20,7 @@ PREFIX = /usr BINDIR = $(PREFIX)/bin USR_BIN := \ proxmox-tui-installer\ + proxmox-fetch-answer\ proxmox-auto-installer COMPILED_BINS := \ diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs new file mode 100644 index 0000000..9e89a3c --- /dev/null +++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs @@ -0,0 +1,71 @@ +use anyhow::{anyhow, Error, Result}; +use log::{error, info, LevelFilter}; +use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger}; +use std::io::Write; +use std::process::{Command, ExitCode, Stdio}; + +static LOGGER: AutoInstLogger = AutoInstLogger; + +pub fn init_log() -> Result<()> { + AutoInstLogger::init("/tmp/fetch_answer.log")?; + log::set_logger(&LOGGER) + .map(|()| log::set_max_level(LevelFilter::Info)) + .map_err(|err| anyhow!(err)) +} + +fn fetch_answer() -> Result<String> { + match FetchFromPartition::get_answer() { + Ok(answer) => return Ok(answer), + Err(err) => info!("Fetching answer file from partition failed: {err}"), + } + // TODO: add more options to get an answer file, e.g. download from url where url could be + // fetched via txt records on predefined subdomain, kernel param, dhcp option, ... + + Err(Error::msg("Could not find any answer file!")) +} + +fn main() -> ExitCode { + if let Err(err) = init_log() { + panic!("could not initilize logging: {err}"); + } + + info!("Fetching answer file"); + let answer = match fetch_answer() { + Ok(answer) => answer, + Err(err) => { + error!("Aborting: {}", err); + return ExitCode::FAILURE; + } + }; + + let mut child = match Command::new("proxmox-auto-installer") + .stdout(Stdio::inherit()) + .stdin(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + { + Ok(child) => child, + Err(err) => panic!("Failed to start automatic installation: {}", err), + }; + + let mut stdin = child.stdin.take().expect("Failed to open stdin"); + std::thread::spawn(move || { + stdin + .write_all(answer.as_bytes()) + .expect("Failed to write to stdin"); + }); + + match child.wait() { + Ok(status) => { + if status.success() { + ExitCode::SUCCESS + } else { + ExitCode::FAILURE // Will be trapped + } + } + Err(err) => { + error!("Auto installer exited: {err}"); + ExitCode::FAILURE + } + } +} diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs new file mode 100644 index 0000000..11d6937 --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs @@ -0,0 +1,2 @@ +pub mod partition; +mod utils; diff --git a/proxmox-auto-installer/src/fetch_plugins/partition.rs b/proxmox-auto-installer/src/fetch_plugins/partition.rs new file mode 100644 index 0000000..0c47a62 --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/partition.rs @@ -0,0 +1,32 @@ +use anyhow::{Error, Result}; +use std::{fs::read_to_string, path::Path}; +use log::info; + +use crate::fetch_plugins::utils::mount_proxmoxinst_part; + +static ANSWER_FILE: &str = "answer.toml"; + +pub struct FetchFromPartition; + +impl FetchFromPartition { + /// Returns the contents of the answer file + pub fn get_answer() -> Result<String> { + info!("Checking for answer file on partition."); + let mount_path = mount_proxmoxinst_part()?; + let answer = Self::get_answer_file(&mount_path)?; + info!("Found answer file on partition."); + Ok(answer) + } + + /// Searches for answer file and returns contents if found + fn get_answer_file(mount_path: &str) -> Result<String> { + let answer_path = Path::new(mount_path).join(ANSWER_FILE); + match answer_path.try_exists() { + Ok(true) => Ok(read_to_string(answer_path)?), + _ => Err(Error::msg(format!( + "could not find answer file expected at: {}", + answer_path.display() + ))), + } + } +} diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils.rs new file mode 100644 index 0000000..82cd3e0 --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/utils.rs @@ -0,0 +1,90 @@ +use anyhow::{bail, Result}; +use log::{info, warn}; +use std::{ + fs::create_dir_all, + path::{Path, PathBuf}, + process::Command, +}; + +/// Searches for upper and lower case existence of the partlabel in the search_path +/// +/// # Arguemnts +/// * `partlabel_lower` - Partition Label in lower case +/// * `search_path` - Path where to search for the partiiton label +/// search_path: String +pub fn scan_partlabels(partlabel_lower: &str, search_path: &str) -> Result<PathBuf> { + let partlabel = partlabel_lower.to_uppercase(); + let path = Path::new(search_path).join(partlabel.clone()); + match path.try_exists() { + Ok(true) => { + info!("Found partition with label '{}'", partlabel); + return Ok(path); + } + Ok(false) => info!("Did not detect partition with label '{}'", partlabel), + Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), + } + + let partlabel = partlabel_lower.to_lowercase(); + let path = Path::new(search_path).join(partlabel.clone()); + match path.try_exists() { + Ok(true) => { + info!("Found partition with label '{}'", partlabel); + return Ok(path); + } + Ok(false) => info!("Did not detect partition with label '{}'", partlabel), + Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), + } + bail!( + "Could not detect upper or lower case labels for '{}'", + partlabel_lower + ); +} + +/// Will mount source path to target_path +/// +/// # Arguments +/// * `source` - `PathBuf` of the source location +/// * `target_path` - Location where to mount, will be created +pub fn mount_part(source: PathBuf, target_path: &str) -> Result<()> { + info!("Mounting partition at {target_path}"); + // create dir for mountpoint + create_dir_all(target_path)?; + match Command::new("mount") + .args(["-o", "ro"]) + .arg(source) + .arg(target_path) + .output() + { + Ok(output) => { + if output.status.success() { + Ok(()) + } else { + warn!("Error mounting: {}", String::from_utf8(output.stderr)?); + Ok(()) + } + } + Err(err) => bail!("Error mounting: {}", err), + } +} + +/// Tries to unmount the specified path. Will warn on errors, but not fail. +/// +/// # Arguemnts +/// * `target_path` - path to unmount +pub fn umount_part(target_path: &str) -> Result<()> { + info!("Unmounting partitiona at {target_path}"); + match Command::new("umount").arg(target_path).output() { + Ok(output) => { + if output.status.success() { + Ok(()) + } else { + warn!("Error unmounting: {}", String::from_utf8(output.stderr)?); + Ok(()) + } + } + Err(err) => { + warn!("Error unmounting: {}", err); + Ok(()) + } + } +} diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs index 6636cc7..0a153b2 100644 --- a/proxmox-auto-installer/src/lib.rs +++ b/proxmox-auto-installer/src/lib.rs @@ -1,4 +1,5 @@ pub mod answer; +pub mod fetch_plugins; pub mod log; pub mod udevinfo; pub mod utils; -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 14/22] unconfigured: add proxauto as option to start auto installer 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (12 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 13/22] auto-installer: add fetch answer binary Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 15/22] auto-installer: use glob crate for pattern matching Aaron Lauterer ` (9 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- unconfigured.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/unconfigured.sh b/unconfigured.sh index cf7de83..f668be4 100755 --- a/unconfigured.sh +++ b/unconfigured.sh @@ -5,6 +5,7 @@ trap "err_reboot" ERR # NOTE: we nowadays get exec'd by the initrd's PID 1, so we're the new PID 1 parse_cmdline() { + proxauto=0 proxdebug=0 proxtui=0 serial=0 @@ -17,6 +18,9 @@ parse_cmdline() { proxtui) proxtui=1 ;; + proxauto) + proxauto=1 + ;; console=ttyS*) serial=1 ;; @@ -222,6 +226,10 @@ if [ $proxtui -ne 0 ]; then echo "Starting the TUI installer" /usr/bin/proxmox-tui-installer 2>/dev/tty2 +elif [ $proxauto -ne 0 ]; then + /usr/bin/proxmox-low-level-installer dump-udev + echo "Starting automatic installation" + /usr/bin/proxmox-fetch-answer else echo "Starting the installer GUI - see tty2 (CTRL+ALT+F2) for any errors..." xinit -- -dpi "$DPI" -s 0 >/dev/tty2 2>&1 -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 15/22] auto-installer: use glob crate for pattern matching 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (13 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 14/22] unconfigured: add proxauto as option to start auto installer Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 16/22] auto-installer: utils: make get_udev_index functions public Aaron Lauterer ` (8 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/Cargo.toml | 1 + proxmox-auto-installer/src/utils.rs | 48 +++++++++++------------------ 2 files changed, 19 insertions(+), 30 deletions(-) diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 3dcb5fa..c00d25a 100644 --- a/proxmox-auto-installer/Cargo.toml +++ b/proxmox-auto-installer/Cargo.toml @@ -9,6 +9,7 @@ homepage = "https://www.proxmox.com" [dependencies] anyhow = "1.0" +glob = "0.3" proxmox-installer-common = { path = "../proxmox-installer-common" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs index 23bedd9..38aab91 100644 --- a/proxmox-auto-installer/src/utils.rs +++ b/proxmox-auto-installer/src/utils.rs @@ -1,4 +1,5 @@ use anyhow::{anyhow, bail, Context, Result}; +use glob::Pattern; use log::info; use std::{ collections::BTreeMap, @@ -21,30 +22,12 @@ use proxmox_installer_common::{ }; use serde::Deserialize; -/// Supports the globbing character '*' at the beginning, end or both of the pattern. -/// Globbing within the pattern is not supported -fn find_with_glob(pattern: &str, value: &str) -> bool { - let globbing_symbol = '*'; - let mut start_glob = false; - let mut end_glob = false; - let mut pattern = pattern; - - if pattern.starts_with(globbing_symbol) { - start_glob = true; - pattern = &pattern[1..]; - } - - if pattern.ends_with(globbing_symbol) { - end_glob = true; - pattern = &pattern[..pattern.len() - 1] - } - - match (start_glob, end_glob) { - (true, true) => value.contains(pattern), - (true, false) => value.ends_with(pattern), - (false, true) => value.starts_with(pattern), - _ => value == pattern, - } +fn find_with_glob(pattern: &str, value: &str) -> Result<bool> { + let p = Pattern::new(pattern)?; + match p.matches(value) { + true => Ok(true), + false => Ok(false), + } } pub fn get_network_settings( @@ -107,7 +90,7 @@ fn get_single_udev_index( 'outer: for (dev, dev_values) in udev_list { for (filter_key, filter_value) in &filter { for (udev_key, udev_value) in dev_values { - if udev_key == filter_key && find_with_glob(filter_value, udev_value) { + if udev_key == filter_key && find_with_glob(filter_value, udev_value)? { dev_index = Some(dev.clone()); break 'outer; // take first match } @@ -132,7 +115,7 @@ fn get_matched_udev_indexes( let mut did_match_all = true; for (filter_key, filter_value) in &filter { for (udev_key, udev_value) in dev_values { - if udev_key == filter_key && find_with_glob(filter_value, udev_value) { + if udev_key == filter_key && find_with_glob(filter_value, udev_value)? { did_match_once = true; } else if udev_key == filter_key { did_match_all = false; @@ -463,9 +446,14 @@ mod tests { #[test] fn test_glob_patterns() { let test_value = "foobar"; - assert_eq!(find_with_glob("*bar", test_value), true); - assert_eq!(find_with_glob("foo*", test_value), true); - assert_eq!(find_with_glob("foobar", test_value), true); - assert_eq!(find_with_glob("oobar", test_value), false); + assert_eq!(find_with_glob("*bar", test_value).unwrap(), true); + assert_eq!(find_with_glob("foo*", test_value).unwrap(), true); + assert_eq!(find_with_glob("foobar", test_value).unwrap(), true); + assert_eq!(find_with_glob("oobar", test_value).unwrap(), false); + assert_eq!(find_with_glob("f*bar", test_value).unwrap(), true); + assert_eq!(find_with_glob("f?bar", test_value).unwrap(), false); + assert_eq!(find_with_glob("fo?bar", test_value).unwrap(), true); + assert_eq!(find_with_glob("f[!a]obar", test_value).unwrap(), true); + assert_eq!(find_with_glob("f[oa]obar", test_value).unwrap(), true); } } -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 16/22] auto-installer: utils: make get_udev_index functions public 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (14 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 15/22] auto-installer: use glob crate for pattern matching Aaron Lauterer @ 2024-02-21 11:07 ` Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 17/22] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer ` (7 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:07 UTC (permalink / raw) To: pve-devel because we will need to access them directly in the future from a separate binary Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/src/utils.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs index 38aab91..f4fcd29 100644 --- a/proxmox-auto-installer/src/utils.rs +++ b/proxmox-auto-installer/src/utils.rs @@ -79,7 +79,7 @@ pub fn get_network_settings( Ok(network_options) } -fn get_single_udev_index( +pub fn get_single_udev_index( filter: BTreeMap<String, String>, udev_list: &BTreeMap<String, BTreeMap<String, String>>, ) -> Result<String> { @@ -104,7 +104,7 @@ fn get_single_udev_index( Ok(dev_index.unwrap()) } -fn get_matched_udev_indexes( +pub fn get_matched_udev_indexes( filter: BTreeMap<String, String>, udev_list: &BTreeMap<String, BTreeMap<String, String>>, match_all: bool, -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 17/22] auto-installer: add proxmox-autoinst-helper tool 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (15 preceding siblings ...) 2024-02-21 11:07 ` [pve-devel] [PATCH v2 16/22] auto-installer: utils: make get_udev_index functions public Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer ` (6 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel It can parse an answer file to check against syntax errors, test match filters against the current hardware and list properties of the current hardware to match against. Since this tool should be able to run outside of the installer environment, it does not rely on the device information provided by the low-level installer. It instead fetches the list of disks and NICs by itself. The rules when a device is ignored, should match how the low-level installer handles it. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- Makefile | 1 + proxmox-auto-installer/Cargo.toml | 2 + proxmox-auto-installer/src/answer.rs | 3 +- .../src/bin/proxmox-autoinst-helper.rs | 335 ++++++++++++++++++ 4 files changed, 340 insertions(+), 1 deletion(-) create mode 100644 proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs diff --git a/Makefile b/Makefile index b724789..dec5b7b 100644 --- a/Makefile +++ b/Makefile @@ -20,6 +20,7 @@ PREFIX = /usr BINDIR = $(PREFIX)/bin USR_BIN := \ proxmox-tui-installer\ + proxmox-autoinst-helper\ proxmox-fetch-answer\ proxmox-auto-installer diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index c00d25a..52bf004 100644 --- a/proxmox-auto-installer/Cargo.toml +++ b/proxmox-auto-installer/Cargo.toml @@ -9,9 +9,11 @@ homepage = "https://www.proxmox.com" [dependencies] anyhow = "1.0" +clap = { version = "4.0", features = ["derive"] } glob = "0.3" proxmox-installer-common = { path = "../proxmox-installer-common" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" toml = "0.7" log = "0.4.20" +regex = "1.7" diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs index 8ba677a..2485726 100644 --- a/proxmox-auto-installer/src/answer.rs +++ b/proxmox-auto-installer/src/answer.rs @@ -1,3 +1,4 @@ +use clap::ValueEnum; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; @@ -44,7 +45,7 @@ pub struct Disks { pub btrfs: Option<BtrfsOptions>, } -#[derive(Clone, Deserialize, Debug, PartialEq)] +#[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)] #[serde(rename_all = "lowercase")] pub enum FilterMatch { Any, diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs new file mode 100644 index 0000000..9be7876 --- /dev/null +++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs @@ -0,0 +1,335 @@ +use anyhow::{bail, Result}; +use clap::{Args, Parser, Subcommand, ValueEnum}; +use glob::Pattern; +use regex::Regex; +use serde::{Deserialize, Serialize}; +use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command}; + +use proxmox_auto_installer::{ + answer::Answer, + answer::FilterMatch, + utils::{get_matched_udev_indexes, get_single_udev_index}, +}; + +/// This tool validates the format of an answer file. Additionally it can test match filters and +/// print information on the properties to match against for the current hardware. +#[derive(Parser, Debug)] +#[command(author, version, about, long_about = None)] +struct Cli { + #[command(subcommand)] + command: Commands, +} + +#[derive(Subcommand, Debug)] +enum Commands { + Answer(CommandAnswer), + Match(CommandMatch), + Info(CommandInfo), +} + +/// Show device information that can be used for filters +#[derive(Args, Debug)] +struct CommandInfo { + /// For which device type information should be shown + #[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)] + device: AllDeviceTypes, +} + +/// Test which devices the given filter matches against +/// +/// Filters support the following syntax: +/// ? Match a single character +/// * Match any number of characters +/// [a], [0-9] Specifc character or range of characters +/// [!a] Negate a specific character of range +/// +/// To avoid globbing characters being interpreted by the shell, use single quotes. +/// Multiple filters can be defined. +/// +/// Examples: +/// Match disks against the serial number and device name, both must match: +/// +/// proxmox-installer-filter match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*' +#[derive(Args, Debug)] +#[command(verbatim_doc_comment)] +struct CommandMatch { + /// Device type to match the filter against + r#type: Devicetype, + + /// Filter in the format KEY=VALUE where the key is the UDEV key and VALUE the filter string. + /// Multiple filters are possible, separated by a space. + filter: Vec<String>, + + /// Defines if any filter or all filters must match. + #[arg(long, value_enum, default_value_t=FilterMatch::Any)] + filter_match: FilterMatch, +} + +/// Validate if an answer file is formatted correctly. +#[derive(Args, Debug)] +struct CommandAnswer { + /// Path to the answer file + path: PathBuf, +} + +#[derive(Args, Debug)] +struct GlobalOpts { + /// Output format + #[arg(long, short, value_enum)] + format: OutputFormat, +} + +#[derive(Clone, Debug, ValueEnum, PartialEq)] +enum AllDeviceTypes { + All, + Network, + Disk, +} + +#[derive(Clone, Debug, ValueEnum)] +enum Devicetype { + Network, + Disk, +} + +#[derive(Clone, Debug, ValueEnum)] +enum OutputFormat { + Pretty, + Json, +} + +#[derive(Serialize)] +struct Devs { + disks: Option<BTreeMap<String, BTreeMap<String, String>>>, + nics: Option<BTreeMap<String, BTreeMap<String, String>>>, +} + +fn main() { + let args = Cli::parse(); + let res = match &args.command { + Commands::Info(args) => info(args), + Commands::Match(args) => match_filter(args), + Commands::Answer(args) => validate_answer(args), + }; + if let Err(err) = res { + eprintln!("{err}"); + std::process::exit(1); + } +} + +fn info(args: &CommandInfo) -> Result<()> { + let mut devs = Devs { + disks: None, + nics: None, + }; + + if args.device == AllDeviceTypes::Network || args.device == AllDeviceTypes::All { + match get_nics() { + Ok(res) => devs.nics = Some(res), + Err(err) => bail!("Error getting NIC data: {err}"), + } + } + if args.device == AllDeviceTypes::Disk || args.device == AllDeviceTypes::All { + match get_disks() { + Ok(res) => devs.disks = Some(res), + Err(err) => bail!("Error getting disk data: {err}"), + } + } + println!("{}", serde_json::to_string_pretty(&devs).unwrap()); + Ok(()) +} + +fn match_filter(args: &CommandMatch) -> Result<()> { + let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type { + Devicetype::Disk => get_disks().unwrap(), + Devicetype::Network => get_nics().unwrap(), + }; + // parse filters + + let mut filters: BTreeMap<String, String> = BTreeMap::new(); + + for f in &args.filter { + match f.split_once('=') { + Some((key, value)) => { + if key.is_empty() || value.is_empty() { + bail!("Filter key or value is empty in filter: '{f}'"); + } + filters.insert(String::from(key), String::from(value)); + } + None => { + bail!("Could not find separator '=' in filter: '{f}'"); + } + } + } + + // align return values + let result = match args.r#type { + Devicetype::Disk => { + get_matched_udev_indexes(filters, &devs, args.filter_match == FilterMatch::All) + } + Devicetype::Network => get_single_udev_index(filters, &devs).map(|r| vec![r]), + }; + + match result { + Ok(result) => println!("{}", serde_json::to_string_pretty(&result).unwrap()), + Err(err) => bail!("Error matching filters: {err}"), + } + Ok(()) +} + +fn validate_answer(args: &CommandAnswer) -> Result<()> { + let mut file = match fs::File::open(&args.path) { + Ok(file) => file, + Err(err) => bail!( + "Opening answer file '{}' failed: {err}", + args.path.display() + ), + }; + let mut contents = String::new(); + if let Err(err) = file.read_to_string(&mut contents) { + bail!("Reading from file '{}' failed: {err}", args.path.display()); + } + + let _answer: Answer = match toml::from_str(&contents) { + Ok(answer) => { + println!("The file was parsed successfully, no syntax errors found!"); + answer + } + Err(err) => bail!("Error parsing answer file: {err}"), + }; + Ok(()) +} + +fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> { + let unwantend_block_devs = vec![ + "ram[0-9]*", + "loop[0-9]*", + "md[0-9]*", + "dm-*", + "fd[0-9]*", + "sr[0-9]*", + ]; + + // compile Regex here once and not inside the loop + let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?; + let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?; + let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?; + + let re_name = Regex::new(r"(?m)^N: (.*)$")?; + let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?; + + let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); + + 'outer: for entry in fs::read_dir("/sys/block")? { + let entry = entry.unwrap(); + let filename = entry.file_name().into_string().unwrap(); + + for p in &unwantend_block_devs { + if Pattern::new(p)?.matches(&filename) { + continue 'outer; + } + } + + let output = match get_udev_properties(&entry.path()) { + Ok(output) => output, + Err(err) => { + eprint!("{err}"); + continue 'outer; + } + }; + + if !re_disk.is_match(&output) { + continue 'outer; + }; + if re_cdrom.is_match(&output) { + continue 'outer; + }; + if re_iso9660.is_match(&output) { + continue 'outer; + }; + + let mut name = filename; + if let Some(cap) = re_name.captures(&output) { + if let Some(res) = cap.get(1) { + name = String::from(res.as_str()); + } + } + + let mut udev_props: BTreeMap<String, String> = BTreeMap::new(); + + for line in output.lines() { + if let Some(caps) = re_props.captures(line) { + let key = String::from(caps.get(1).unwrap().as_str()); + let value = String::from(caps.get(2).unwrap().as_str()); + udev_props.insert(key, value); + } + } + + disks.insert(name, udev_props); + } + Ok(disks) +} + +#[derive(Deserialize, Debug)] +struct IpLinksInfo { + ifname: String, +} + +fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> { + let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?; + let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); + + let ip_output = Command::new("/usr/sbin/ip") + .arg("-j") + .arg("link") + .output()?; + let parsed_links: Vec<IpLinksInfo> = + serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?; + let mut links: Vec<String> = Vec::new(); + + for link in parsed_links { + if link.ifname == *"lo" { + continue; + } + links.push(link.ifname); + } + + for link in links { + let path = format!("/sys/class/net/{link}"); + + let output = match get_udev_properties(&PathBuf::from(path)) { + Ok(output) => output, + Err(err) => { + eprint!("{err}"); + continue; + } + }; + + let mut udev_props: BTreeMap<String, String> = BTreeMap::new(); + + for line in output.lines() { + if let Some(caps) = re_props.captures(line) { + let key = String::from(caps.get(1).unwrap().as_str()); + let value = String::from(caps.get(2).unwrap().as_str()); + udev_props.insert(key, value); + } + } + + nics.insert(link, udev_props); + } + Ok(nics) +} + +fn get_udev_properties(path: &PathBuf) -> Result<String> { + let udev_output = Command::new("udevadm") + .arg("info") + .arg("--path") + .arg(path) + .arg("--query") + .arg("all") + .output()?; + if !udev_output.status.success() { + bail!("could not run udevadm successfully for {}", path.display()); + } + Ok(String::from_utf8(udev_output.stdout)?) +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (16 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 17/22] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 14:09 ` Christoph Heiss 2024-02-21 11:08 ` [pve-devel] [PATCH v2 19/22] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer ` (5 subsequent siblings) 23 siblings, 1 reply; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel They will be used as payload when POSTing a request for an answer file. The idea is, that with this information, it should be possible to identify the system and generate a matching answer file on the fly. Many of these properties can also be found on the machine or packaging of the machine and could therefore be scanned into a database. Identifiers are the following properties from `dmidecode` sections 1, 2, and 3: * Asset Tag * Product Name * Serial Number * SKU Number * UUID As well as a list of the MAC addresses of all the NICs. Since we now have more than a simple utils.rs module in the fetch plugins, it, and the additional fetch plugin utilities are placed in their own directory. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- .../src/fetch_plugins/mod.rs | 2 +- .../src/fetch_plugins/utils.rs | 90 -------- .../src/fetch_plugins/utils/mod.rs | 113 ++++++++++ .../src/fetch_plugins/utils/sysinfo.rs | 200 ++++++++++++++++++ 4 files changed, 314 insertions(+), 91 deletions(-) delete mode 100644 proxmox-auto-installer/src/fetch_plugins/utils.rs create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/mod.rs create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs index 11d6937..6f1e8a2 100644 --- a/proxmox-auto-installer/src/fetch_plugins/mod.rs +++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs @@ -1,2 +1,2 @@ pub mod partition; -mod utils; +pub mod utils; diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils.rs deleted file mode 100644 index 82cd3e0..0000000 --- a/proxmox-auto-installer/src/fetch_plugins/utils.rs +++ /dev/null @@ -1,90 +0,0 @@ -use anyhow::{bail, Result}; -use log::{info, warn}; -use std::{ - fs::create_dir_all, - path::{Path, PathBuf}, - process::Command, -}; - -/// Searches for upper and lower case existence of the partlabel in the search_path -/// -/// # Arguemnts -/// * `partlabel_lower` - Partition Label in lower case -/// * `search_path` - Path where to search for the partiiton label -/// search_path: String -pub fn scan_partlabels(partlabel_lower: &str, search_path: &str) -> Result<PathBuf> { - let partlabel = partlabel_lower.to_uppercase(); - let path = Path::new(search_path).join(partlabel.clone()); - match path.try_exists() { - Ok(true) => { - info!("Found partition with label '{}'", partlabel); - return Ok(path); - } - Ok(false) => info!("Did not detect partition with label '{}'", partlabel), - Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), - } - - let partlabel = partlabel_lower.to_lowercase(); - let path = Path::new(search_path).join(partlabel.clone()); - match path.try_exists() { - Ok(true) => { - info!("Found partition with label '{}'", partlabel); - return Ok(path); - } - Ok(false) => info!("Did not detect partition with label '{}'", partlabel), - Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), - } - bail!( - "Could not detect upper or lower case labels for '{}'", - partlabel_lower - ); -} - -/// Will mount source path to target_path -/// -/// # Arguments -/// * `source` - `PathBuf` of the source location -/// * `target_path` - Location where to mount, will be created -pub fn mount_part(source: PathBuf, target_path: &str) -> Result<()> { - info!("Mounting partition at {target_path}"); - // create dir for mountpoint - create_dir_all(target_path)?; - match Command::new("mount") - .args(["-o", "ro"]) - .arg(source) - .arg(target_path) - .output() - { - Ok(output) => { - if output.status.success() { - Ok(()) - } else { - warn!("Error mounting: {}", String::from_utf8(output.stderr)?); - Ok(()) - } - } - Err(err) => bail!("Error mounting: {}", err), - } -} - -/// Tries to unmount the specified path. Will warn on errors, but not fail. -/// -/// # Arguemnts -/// * `target_path` - path to unmount -pub fn umount_part(target_path: &str) -> Result<()> { - info!("Unmounting partitiona at {target_path}"); - match Command::new("umount").arg(target_path).output() { - Ok(output) => { - if output.status.success() { - Ok(()) - } else { - warn!("Error unmounting: {}", String::from_utf8(output.stderr)?); - Ok(()) - } - } - Err(err) => { - warn!("Error unmounting: {}", err); - Ok(()) - } - } -} diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs new file mode 100644 index 0000000..37341ee --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs @@ -0,0 +1,113 @@ +use anyhow::{Error, Result}; +use log::{info, warn}; +use serde::Deserialize; +use serde_json; +use std::{ + fs::{self, create_dir_all}, + path::{Path, PathBuf}, + process::Command, +}; + +static ANSWER_MP: &str = "/mnt/answer"; +static PARTLABEL: &str = "proxmoxinst"; +static SEARCH_PATH: &str = "/dev/disk/by-label"; + +pub mod sysinfo; + +/// Searches for upper and lower case existence of the partlabel in the search_path +/// +/// # Arguemnts +/// * `partlabel_lower` - Partition Label in lower case +/// * `search_path` - Path where to search for the partiiton label +/// search_path: String +pub fn scan_partlabels(partlabel_lower: &str, search_path: &str) -> Result<PathBuf> { + let partlabel = partlabel_lower.to_uppercase(); + let path = Path::new(search_path).join(partlabel.clone()); + match path.try_exists() { + Ok(true) => { + info!("Found partition with label '{}'", partlabel); + return Ok(path); + } + Ok(false) => info!("Did not detect partition with label '{}'", partlabel), + Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), + } + + let partlabel = partlabel_lower.to_lowercase(); + let path = Path::new(search_path).join(partlabel.clone()); + match path.try_exists() { + Ok(true) => { + info!("Found partition with label '{}'", partlabel); + return Ok(path); + } + Ok(false) => info!("Did not detect partition with label '{}'", partlabel), + Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err), + } + Err(Error::msg(format!( + "Could not detect upper or lower case labels for '{partlabel_lower}'" + ))) +} + +/// Will search and mount a partition/FS labeled proxmoxinst in lower or uppercase to ANSWER_MP; +pub fn mount_proxmoxinst_part() -> Result<String> { + if let Ok(true) = check_if_mounted(ANSWER_MP) { + info!("Skipping: '{ANSWER_MP}' is already mounted."); + return Ok(ANSWER_MP.into()); + } + let part_path = scan_partlabels(PARTLABEL, SEARCH_PATH)?; + info!("Mounting partition at {ANSWER_MP}"); + // create dir for mountpoint + create_dir_all(ANSWER_MP)?; + match Command::new("mount") + .args(["-o", "ro"]) + .arg(part_path) + .arg(ANSWER_MP) + .output() + { + Ok(output) => { + if output.status.success() { + Ok(ANSWER_MP.into()) + } else { + warn!("Error mounting: {}", String::from_utf8(output.stderr)?); + Ok(ANSWER_MP.into()) + } + } + Err(err) => Err(Error::msg(format!("Error mounting: {err}"))), + } +} + +fn check_if_mounted(target_path: &str) -> Result<bool> { + let mounts = fs::read_to_string("/proc/mounts")?; + for line in mounts.lines() { + if let Some(mp) = line.split(' ').nth(1) { + if mp == target_path { + return Ok(true); + } + } + } + Ok(false) +} + +#[derive(Deserialize, Debug)] +struct IpLinksUdevInfo { + ifname: String, +} + +/// Returns vec of usable NICs +pub fn get_nic_list() -> Result<Vec<String>> { + let ip_output = Command::new("/usr/sbin/ip") + .arg("-j") + .arg("link") + .output()?; + let parsed_links: Vec<IpLinksUdevInfo> = + serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?; + let mut links: Vec<String> = Vec::new(); + + for link in parsed_links { + if link.ifname == *"lo" { + continue; + } + links.push(link.ifname); + } + + Ok(links) +} diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs new file mode 100644 index 0000000..74701cd --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs @@ -0,0 +1,200 @@ +use anyhow::{bail, Result}; +use serde::Serialize; +use std::{collections::HashMap, fs, process::Command}; + +use super::get_nic_list; + +pub fn get_sysinfo(pretty: bool) -> Result<String> { + let mut system = HashMap::new(); + let mut baseboard = HashMap::new(); + let mut chassis = HashMap::new(); + for option in 1..=3 { + let dmiresult = Command::new("dmidecode") + .arg("-t") + .arg(format!("{option}")) + .output()?; + + if dmiresult.status.success() { + let output = String::from_utf8(dmiresult.stdout)?; + match option { + 1 => system = parse_dmidecode(&output)?, + 2 => baseboard = parse_dmidecode(&output)?, + 3 => chassis = parse_dmidecode(&output)?, + _ => (), + } + } else { + let stderr = String::from_utf8(dmiresult.stderr)?; + bail!("Failed to get dmidecode information. Are you running as root? '{stderr}'"); + } + } + + let mut mac_addresses: Vec<String> = Vec::new(); + let links = get_nic_list()?; + for link in links { + let address = fs::read_to_string(format!("/sys/class/net/{link}/address"))?; + let address = String::from(address.trim()); + mac_addresses.push(address); + } + + let sysinfo = SysInfo { + system, + baseboard, + chassis, + mac_addresses, + }; + if pretty { + return Ok(serde_json::to_string_pretty(&sysinfo)?); + } + Ok(serde_json::to_string(&sysinfo)?) +} + +#[derive(Debug, Serialize)] +struct SysInfo { + system: HashMap<String, String>, + baseboard: HashMap<String, String>, + chassis: HashMap<String, String>, + mac_addresses: Vec<String>, +} + +fn parse_dmidecode(output: &str) -> Result<HashMap<String, String>> { + let keywords = vec![ + "Asset Tag", + "Product Name", + "Serial Number", + "SKU Number", + "UUID", + ]; + + let mut res: HashMap<String, String> = HashMap::new(); + for mut line in output.lines() { + line = line.trim(); + if let Some((key, value)) = line.split_once(':') { + if keywords.contains(&key) { + res.insert(String::from(key), String::from(value.trim())); + } + } + } + + Ok(res) +} + +#[cfg(test)] +mod tests { + use std::collections::HashMap; + + use super::parse_dmidecode; + + #[test] + fn dmidecode_parse() { + let system1 = String::from( + r#" +# dmidecode 3.4 +Getting SMBIOS data from sysfs. +SMBIOS 3.2.0 present. + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: GIGABYTE + Product Name: MZ32-AR0-00 + Version: 0100 + Serial Number: 01234567890123456789AB + UUID: 61df0000-9855-11ed-8000-b42e99acXXXX + Wake-up Type: Power Switch + SKU Number: 01234567890123456789AB + Family: Server"#, + ); + + let mut system1_check: HashMap<String, String> = HashMap::new(); + system1_check.insert( + String::from("Serial Number"), + String::from("01234567890123456789AB"), + ); + system1_check.insert( + String::from("UUID"), + String::from("61df0000-9855-11ed-8000-b42e99acXXXX"), + ); + system1_check.insert( + String::from("SKU Number"), + String::from("01234567890123456789AB"), + ); + system1_check.insert( + String::from("Product Name"), + String::from("MZ32-AR0-00"), + ); + + let baseboard1 = String::from( + r#" +# dmidecode 3.4 +Getting SMBIOS data from sysfs. +SMBIOS 3.2.0 present. + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: GIGABYTE + Product Name: MZ32-AR0-00 + Version: 01000100 + Serial Number: JGBNA600XXX + Asset Tag: 01234567890123456789AB + Features: + Board is a hosting board + Board is removable + Board is replaceable + Location In Chassis: 01234567890123456789AB + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0"#, + ); + let mut baseboard1_check: HashMap<String, String> = HashMap::new(); + baseboard1_check.insert(String::from("Serial Number"), String::from("JGBNA600XXX")); + baseboard1_check.insert( + String::from("Asset Tag"), + String::from("01234567890123456789AB"), + ); + baseboard1_check.insert( + String::from("Product Name"), + String::from("MZ32-AR0-00"), + ); + + let chassis1 = String::from( + r#" +# dmidecode 3.4 +Getting SMBIOS data from sysfs. +SMBIOS 3.2.0 present. + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: GIGABYTE + Type: Main Server Chassis + Lock: Not Present + Version: 01234567 + Serial Number: 01234567890123456789AB + Asset Tag: 01234567890123456789AB + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: 01234567890123456789AB"#, + ); + let mut chassis1_check: HashMap<String, String> = HashMap::new(); + chassis1_check.insert( + String::from("Serial Number"), + String::from("01234567890123456789AB"), + ); + chassis1_check.insert( + String::from("Asset Tag"), + String::from("01234567890123456789AB"), + ); + chassis1_check.insert( + String::from("SKU Number"), + String::from("01234567890123456789AB"), + ); + + assert_eq!(parse_dmidecode(&system1).unwrap(), system1_check); + assert_eq!(parse_dmidecode(&baseboard1).unwrap(), baseboard1_check); + assert_eq!(parse_dmidecode(&chassis1).unwrap(), chassis1_check); + } +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code 2024-02-21 11:08 ` [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer @ 2024-02-21 14:09 ` Christoph Heiss 2024-02-21 16:07 ` Aaron Lauterer 0 siblings, 1 reply; 29+ messages in thread From: Christoph Heiss @ 2024-02-21 14:09 UTC (permalink / raw) To: Aaron Lauterer; +Cc: Proxmox VE development discussion On Wed, Feb 21, 2024 at 12:08:01PM +0100, Aaron Lauterer wrote: > They will be used as payload when POSTing a request for an answer file. The > idea is, that with this information, it should be possible to identify > the system and generate a matching answer file on the fly. > Many of these properties can also be found on the machine or packaging > of the machine and could therefore be scanned into a database. > > Identifiers are the following properties from `dmidecode` sections 1, 2, > and 3: > * Asset Tag > * Product Name > * Serial Number > * SKU Number > * UUID > > As well as a list of the MAC addresses of all the NICs. [..] > diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs > new file mode 100644 > index 0000000..74701cd > --- /dev/null > +++ b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs > @@ -0,0 +1,200 @@ > +use anyhow::{bail, Result}; > +use serde::Serialize; > +use std::{collections::HashMap, fs, process::Command}; > + > +use super::get_nic_list; > + > +pub fn get_sysinfo(pretty: bool) -> Result<String> { While looking at this and reading the `dmidecode` manpage a bit, it looks like everything collected here is also available under /sys/devices/virtual/dmi/id as separate files, e.g. # ls /sys/devices/virtual/dmi/id/ bios_date chassis_asset_tag product_name bios_release chassis_serial product_serial bios_vendor chassis_type product_sku bios_version chassis_vendor product_uuid board_asset_tag chassis_version product_version board_name ec_firmware_release subsystem board_serial modalias sys_vendor board_vendor power uevent board_version product_family Most of these file are even world-readable, just some (e.g. *_serial) need root to read. So it could be a simple list of filenames to read from there. So that could be a nice alternative to calling and parsing the `dmidecode` output. The names of the files would also serve as good keys for matching. Maybe you already considered it? Just wanted to throw this in there. Not everything seems to be available there, e.g. "Boot-up State", "Power Supply State" and "Thermal State", but at least everything which is interesting to identify machines uniquely seems to be there. > + let mut system = HashMap::new(); > + let mut baseboard = HashMap::new(); > + let mut chassis = HashMap::new(); > + for option in 1..=3 { > + let dmiresult = Command::new("dmidecode") > + .arg("-t") > + .arg(format!("{option}")) > + .output()?; > + > + if dmiresult.status.success() { > + let output = String::from_utf8(dmiresult.stdout)?; > + match option { > + 1 => system = parse_dmidecode(&output)?, > + 2 => baseboard = parse_dmidecode(&output)?, > + 3 => chassis = parse_dmidecode(&output)?, > + _ => (), > + } > + } else { > + let stderr = String::from_utf8(dmiresult.stderr)?; > + bail!("Failed to get dmidecode information. Are you running as root? '{stderr}'"); > + } > + } > + > + let mut mac_addresses: Vec<String> = Vec::new(); > + let links = get_nic_list()?; > + for link in links { > + let address = fs::read_to_string(format!("/sys/class/net/{link}/address"))?; > + let address = String::from(address.trim()); > + mac_addresses.push(address); > + } > + > + let sysinfo = SysInfo { > + system, > + baseboard, > + chassis, > + mac_addresses, > + }; > + if pretty { > + return Ok(serde_json::to_string_pretty(&sysinfo)?); > + } > + Ok(serde_json::to_string(&sysinfo)?) > +} > + > +#[derive(Debug, Serialize)] > +struct SysInfo { > + system: HashMap<String, String>, > + baseboard: HashMap<String, String>, > + chassis: HashMap<String, String>, > + mac_addresses: Vec<String>, > +} > + > +fn parse_dmidecode(output: &str) -> Result<HashMap<String, String>> { > + let keywords = vec![ > + "Asset Tag", > + "Product Name", > + "Serial Number", > + "SKU Number", > + "UUID", > + ]; > + > + let mut res: HashMap<String, String> = HashMap::new(); > + for mut line in output.lines() { > + line = line.trim(); > + if let Some((key, value)) = line.split_once(':') { > + if keywords.contains(&key) { > + res.insert(String::from(key), String::from(value.trim())); > + } > + } > + } > + > + Ok(res) > +} > + > +#[cfg(test)] > +mod tests { > + use std::collections::HashMap; > + > + use super::parse_dmidecode; > + > + #[test] > + fn dmidecode_parse() { > + let system1 = String::from( > + r#" > +# dmidecode 3.4 > +Getting SMBIOS data from sysfs. > +SMBIOS 3.2.0 present. > + > +Handle 0x0001, DMI type 1, 27 bytes > +System Information > + Manufacturer: GIGABYTE > + Product Name: MZ32-AR0-00 > + Version: 0100 > + Serial Number: 01234567890123456789AB > + UUID: 61df0000-9855-11ed-8000-b42e99acXXXX > + Wake-up Type: Power Switch > + SKU Number: 01234567890123456789AB > + Family: Server"#, > + ); > + > + let mut system1_check: HashMap<String, String> = HashMap::new(); > + system1_check.insert( > + String::from("Serial Number"), > + String::from("01234567890123456789AB"), > + ); > + system1_check.insert( > + String::from("UUID"), > + String::from("61df0000-9855-11ed-8000-b42e99acXXXX"), > + ); > + system1_check.insert( > + String::from("SKU Number"), > + String::from("01234567890123456789AB"), > + ); > + system1_check.insert( > + String::from("Product Name"), > + String::from("MZ32-AR0-00"), > + ); > + > + let baseboard1 = String::from( > + r#" > +# dmidecode 3.4 > +Getting SMBIOS data from sysfs. > +SMBIOS 3.2.0 present. > + > +Handle 0x0002, DMI type 2, 15 bytes > +Base Board Information > + Manufacturer: GIGABYTE > + Product Name: MZ32-AR0-00 > + Version: 01000100 > + Serial Number: JGBNA600XXX > + Asset Tag: 01234567890123456789AB > + Features: > + Board is a hosting board > + Board is removable > + Board is replaceable > + Location In Chassis: 01234567890123456789AB > + Chassis Handle: 0x0003 > + Type: Motherboard > + Contained Object Handles: 0"#, > + ); > + let mut baseboard1_check: HashMap<String, String> = HashMap::new(); > + baseboard1_check.insert(String::from("Serial Number"), String::from("JGBNA600XXX")); > + baseboard1_check.insert( > + String::from("Asset Tag"), > + String::from("01234567890123456789AB"), > + ); > + baseboard1_check.insert( > + String::from("Product Name"), > + String::from("MZ32-AR0-00"), > + ); > + > + let chassis1 = String::from( > + r#" > +# dmidecode 3.4 > +Getting SMBIOS data from sysfs. > +SMBIOS 3.2.0 present. > + > +Handle 0x0003, DMI type 3, 22 bytes > +Chassis Information > + Manufacturer: GIGABYTE > + Type: Main Server Chassis > + Lock: Not Present > + Version: 01234567 > + Serial Number: 01234567890123456789AB > + Asset Tag: 01234567890123456789AB > + Boot-up State: Safe > + Power Supply State: Safe > + Thermal State: Safe > + Security Status: None > + OEM Information: 0x00000000 > + Height: Unspecified > + Number Of Power Cords: 1 > + Contained Elements: 0 > + SKU Number: 01234567890123456789AB"#, > + ); > + let mut chassis1_check: HashMap<String, String> = HashMap::new(); > + chassis1_check.insert( > + String::from("Serial Number"), > + String::from("01234567890123456789AB"), > + ); > + chassis1_check.insert( > + String::from("Asset Tag"), > + String::from("01234567890123456789AB"), > + ); > + chassis1_check.insert( > + String::from("SKU Number"), > + String::from("01234567890123456789AB"), > + ); > + > + assert_eq!(parse_dmidecode(&system1).unwrap(), system1_check); > + assert_eq!(parse_dmidecode(&baseboard1).unwrap(), baseboard1_check); > + assert_eq!(parse_dmidecode(&chassis1).unwrap(), chassis1_check); > + } > +} > -- > 2.39.2 > > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > > ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code 2024-02-21 14:09 ` Christoph Heiss @ 2024-02-21 16:07 ` Aaron Lauterer 0 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 16:07 UTC (permalink / raw) To: Christoph Heiss; +Cc: Proxmox VE development discussion On 2/21/24 15:09, Christoph Heiss wrote: > While looking at this and reading the `dmidecode` manpage a bit, it > looks like everything collected here is also available under > /sys/devices/virtual/dmi/id as separate files, e.g. > > # ls /sys/devices/virtual/dmi/id/ > bios_date chassis_asset_tag product_name > bios_release chassis_serial product_serial > bios_vendor chassis_type product_sku > bios_version chassis_vendor product_uuid > board_asset_tag chassis_version product_version > board_name ec_firmware_release subsystem > board_serial modalias sys_vendor > board_vendor power uevent > board_version product_family > > Most of these file are even world-readable, just some (e.g. *_serial) > need root to read. So it could be a simple list of filenames to read > from there. > > So that could be a nice alternative to calling and parsing the > `dmidecode` output. The names of the files would also serve as good keys > for matching. Maybe you already considered it? Just wanted to throw this > in there. > > Not everything seems to be available there, e.g. "Boot-up State", "Power > Supply State" and "Thermal State", but at least everything which is > interesting to identify machines uniquely seems to be there. I did not check where dmidecode gets the info from. Thanks! This sounds even more suitable and easier to implement :) And AFAICT that should be all that is interesting to identify a system. ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 19/22] auto-installer: helper: add subcommand to view indentifiers 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (17 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module Aaron Lauterer ` (4 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel It will collect the information from the current system and show the payload of identifiers that will be send. To avoid confusion, the subcommands for the device info and filter matching have been renamed. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- .../src/bin/proxmox-autoinst-helper.rs | 54 +++++++++---------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs index 9be7876..0ce85b4 100644 --- a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs +++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs @@ -2,12 +2,13 @@ use anyhow::{bail, Result}; use clap::{Args, Parser, Subcommand, ValueEnum}; use glob::Pattern; use regex::Regex; -use serde::{Deserialize, Serialize}; +use serde::Serialize; use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command}; use proxmox_auto_installer::{ answer::Answer, answer::FilterMatch, + fetch_plugins::utils::{sysinfo, get_nic_list}, utils::{get_matched_udev_indexes, get_single_udev_index}, }; @@ -23,13 +24,14 @@ struct Cli { #[derive(Subcommand, Debug)] enum Commands { Answer(CommandAnswer), - Match(CommandMatch), - Info(CommandInfo), + DeviceMatch(CommandDeviceMatch), + DeviceInfo(CommandDeviceInfo), + Identifiers(CommandIdentifiers), } /// Show device information that can be used for filters #[derive(Args, Debug)] -struct CommandInfo { +struct CommandDeviceInfo { /// For which device type information should be shown #[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)] device: AllDeviceTypes, @@ -52,7 +54,7 @@ struct CommandInfo { /// proxmox-installer-filter match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*' #[derive(Args, Debug)] #[command(verbatim_doc_comment)] -struct CommandMatch { +struct CommandDeviceMatch { /// Device type to match the filter against r#type: Devicetype, @@ -72,6 +74,11 @@ struct CommandAnswer { path: PathBuf, } +/// Show identifiers for the current machine. This information is part of the POST request to fetch +/// an answer file. +#[derive(Args, Debug)] +struct CommandIdentifiers {} + #[derive(Args, Debug)] struct GlobalOpts { /// Output format @@ -107,9 +114,10 @@ struct Devs { fn main() { let args = Cli::parse(); let res = match &args.command { - Commands::Info(args) => info(args), - Commands::Match(args) => match_filter(args), + Commands::DeviceInfo(args) => info(args), + Commands::DeviceMatch(args) => match_filter(args), Commands::Answer(args) => validate_answer(args), + Commands::Identifiers(args) => show_identifiers(args), }; if let Err(err) = res { eprintln!("{err}"); @@ -117,7 +125,7 @@ fn main() { } } -fn info(args: &CommandInfo) -> Result<()> { +fn info(args: &CommandDeviceInfo) -> Result<()> { let mut devs = Devs { disks: None, nics: None, @@ -139,7 +147,7 @@ fn info(args: &CommandInfo) -> Result<()> { Ok(()) } -fn match_filter(args: &CommandMatch) -> Result<()> { +fn match_filter(args: &CommandDeviceMatch) -> Result<()> { let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type { Devicetype::Disk => get_disks().unwrap(), Devicetype::Network => get_nics().unwrap(), @@ -200,6 +208,14 @@ fn validate_answer(args: &CommandAnswer) -> Result<()> { Ok(()) } +fn show_identifiers(_args: &CommandIdentifiers) -> Result<()> { + match sysinfo::get_sysinfo(true) { + Ok(res) => println!("{res}"), + Err(err) => eprintln!("Error fetching system identifiers: {err}"), + } + Ok(()) +} + fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> { let unwantend_block_devs = vec![ "ram[0-9]*", @@ -270,30 +286,12 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> { Ok(disks) } -#[derive(Deserialize, Debug)] -struct IpLinksInfo { - ifname: String, -} fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> { let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?; let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); - let ip_output = Command::new("/usr/sbin/ip") - .arg("-j") - .arg("link") - .output()?; - let parsed_links: Vec<IpLinksInfo> = - serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?; - let mut links: Vec<String> = Vec::new(); - - for link in parsed_links { - if link.ifname == *"lo" { - continue; - } - links.push(link.ifname); - } - + let links = get_nic_list()?; for link in links { let path = format!("/sys/class/net/{link}"); -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (18 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 19/22] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 12:21 ` Christoph Heiss 2024-02-21 11:08 ` [pve-devel] [PATCH v2 21/22] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer ` (3 subsequent siblings) 23 siblings, 1 reply; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel It sends a http(s) POST request with the sysinfo as payload and expects an answer file in return. In order to handle non FQDN URLs (e.g. IP addresses) and self signed certificates, it can optionally take an SHA256 fingerprint of the certificate. This can of course also be used to pin a certificate explicitly, even if it would be in the trust chain. A custom cert verifier for ureq / rustl was necessary to get cert fingerprint matching to work. If no fingerprint is proviced, we switch rustls to native-certs and native-tls. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- proxmox-auto-installer/Cargo.toml | 6 ++ .../src/fetch_plugins/utils/mod.rs | 1 + .../src/fetch_plugins/utils/post.rs | 93 +++++++++++++++++++ 3 files changed, 100 insertions(+) create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/post.rs diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 52bf004..8afa2ec 100644 --- a/proxmox-auto-installer/Cargo.toml +++ b/proxmox-auto-installer/Cargo.toml @@ -17,3 +17,9 @@ serde_json = "1.0" toml = "0.7" log = "0.4.20" regex = "1.7" +ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] } +rustls = { version = "0.20", features = [ "dangerous_configuration" ] } +rustls-native-certs = "0.6" +native-tls = "0.2" +sha2 = "0.10" +hex = "0.4" diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs index 37341ee..e4921bf 100644 --- a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs +++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs @@ -12,6 +12,7 @@ static ANSWER_MP: &str = "/mnt/answer"; static PARTLABEL: &str = "proxmoxinst"; static SEARCH_PATH: &str = "/dev/disk/by-label"; +pub mod post; pub mod sysinfo; /// Searches for upper and lower case existence of the partlabel in the search_path diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs new file mode 100644 index 0000000..c9f6ddb --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs @@ -0,0 +1,93 @@ +use anyhow::Result; +use rustls::ClientConfig; +use sha2::{Digest, Sha256}; +use std::sync::Arc; +use ureq::{Agent, AgentBuilder}; + +/// Issues a POST request with the payload (JSON). Optionally a SHA256 fingerprint can be used to +/// check the cert against it, instead of the regular cert validation. +/// To gather the sha256 fingerprint you can use the following command: +/// ```no_compile +/// openssl s_client -connect <host>:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin +/// ``` +/// +/// # Arguemnts +/// * `url` - URL to call +/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional. +/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string. +pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> { + let answer ; + + if let Some(fingerprint) = fingerprint { + let tls_config = ClientConfig::builder() + .with_safe_defaults() + .with_custom_certificate_verifier(VerifyCertFingerprint::new(fingerprint)?) + .with_no_client_auth(); + + let agent: Agent = AgentBuilder::new().tls_config(Arc::new(tls_config)).build(); + + answer = agent + .post(&url) + .set("Content-type", "application/json; charset=utf-") + .send_string(&payload)? + .into_string()?; + } else { + let mut roots = rustls::RootCertStore::empty(); + for cert in rustls_native_certs::load_native_certs()? { + roots.add(&rustls::Certificate(cert.0)).unwrap(); + } + + let tls_config = rustls::ClientConfig::builder() + .with_safe_defaults() + .with_root_certificates(roots) + .with_no_client_auth(); + + let agent = AgentBuilder::new() + .tls_connector(Arc::new(native_tls::TlsConnector::new()?)) + .tls_config(Arc::new(tls_config)) + .build(); + answer = agent + .post(&url) + .set("Content-type", "application/json; charset=utf-") + .send_string(&payload)? + .into_string()?; + } + Ok(answer) +} + +struct VerifyCertFingerprint { + cert_fingerprint: Vec<u8>, +} + +impl VerifyCertFingerprint { + fn new<S: AsRef<str>>(cert_fingerprint: S) -> Result<std::sync::Arc<Self>> { + let cert_fingerprint = cert_fingerprint.as_ref(); + let sanitized = cert_fingerprint.replace(':', ""); + let decoded = hex::decode(sanitized)?; + Ok(std::sync::Arc::new(Self { + cert_fingerprint: decoded, + })) + } +} + +impl rustls::client::ServerCertVerifier for VerifyCertFingerprint { + fn verify_server_cert( + &self, + end_entity: &rustls::Certificate, + _intermediates: &[rustls::Certificate], + _server_name: &rustls::ServerName, + _scts: &mut dyn Iterator<Item = &[u8]>, + _ocsp_response: &[u8], + _now: std::time::SystemTime, + ) -> Result<rustls::client::ServerCertVerified, rustls::Error> { + let mut hasher = Sha256::new(); + hasher.update(end_entity); + let result = hasher.finalize(); + + if result.as_slice() == self.cert_fingerprint { + Ok(rustls::client::ServerCertVerified::assertion()) + } else { + Err(rustls::Error::General("Fingerprint did not match!".into())) + } + } +} -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module 2024-02-21 11:08 ` [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module Aaron Lauterer @ 2024-02-21 12:21 ` Christoph Heiss 0 siblings, 0 replies; 29+ messages in thread From: Christoph Heiss @ 2024-02-21 12:21 UTC (permalink / raw) To: Aaron Lauterer; +Cc: Proxmox VE development discussion On Wed, Feb 21, 2024 at 12:08:03PM +0100, Aaron Lauterer wrote: > It sends a http(s) POST request with the sysinfo as payload and expects > an answer file in return. > [..] > diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs > new file mode 100644 > index 0000000..c9f6ddb > --- /dev/null > +++ b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs > @@ -0,0 +1,93 @@ > +use anyhow::Result; > +use rustls::ClientConfig; > +use sha2::{Digest, Sha256}; > +use std::sync::Arc; > +use ureq::{Agent, AgentBuilder}; > + > +/// Issues a POST request with the payload (JSON). Optionally a SHA256 fingerprint can be used to > +/// check the cert against it, instead of the regular cert validation. > +/// To gather the sha256 fingerprint you can use the following command: > +/// ```no_compile > +/// openssl s_client -connect <host>:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin > +/// ``` > +/// > +/// # Arguemnts > +/// * `url` - URL to call > +/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional. > +/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string. > +pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> { > + let answer ; Bit to much whitespaces? > + > + if let Some(fingerprint) = fingerprint { > + let tls_config = ClientConfig::builder() > + .with_safe_defaults() > + .with_custom_certificate_verifier(VerifyCertFingerprint::new(fingerprint)?) > + .with_no_client_auth(); > + > + let agent: Agent = AgentBuilder::new().tls_config(Arc::new(tls_config)).build(); > + > + answer = agent > + .post(&url) > + .set("Content-type", "application/json; charset=utf-") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Should probably read "application/json; charset=utf-8" I assume? :^) > + .send_string(&payload)? > + .into_string()?; > + } else { > + let mut roots = rustls::RootCertStore::empty(); > + for cert in rustls_native_certs::load_native_certs()? { > + roots.add(&rustls::Certificate(cert.0)).unwrap(); > + } > + > + let tls_config = rustls::ClientConfig::builder() > + .with_safe_defaults() > + .with_root_certificates(roots) > + .with_no_client_auth(); > + > + let agent = AgentBuilder::new() > + .tls_connector(Arc::new(native_tls::TlsConnector::new()?)) > + .tls_config(Arc::new(tls_config)) > + .build(); > + answer = agent > + .post(&url) > + .set("Content-type", "application/json; charset=utf-") .. and same here ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 21/22] auto-installer: fetch: add http plugin to fetch answer 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (19 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 22/22] control: update build depends for auto installer Aaron Lauterer ` (2 subsequent siblings) 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel This plugin will send a HTTP POST request with identifying sysinfo to fetch an answer file. The provided sysinfo can be used to identify the system and generate a matching answer file on demand. The URL to send the request to, can be defined in two ways. Via a custom DHCP option or a TXT record on a predefined subdomain, relative to the search domain received via DHCP. Additionally it is possible to specify a SHA256 SSL fingerprint. This can be useful if a self-signed certificate is used or the URL is using an IP address instead of an FQDN. Even with a trusted cert, it can be used to pin this specific certificate. The certificate fingerprint can either be placed on the `proxmoxinst` partition and needs to be called `cert_fingerprint.txt`, or it can be provided in a second custom DHCP option or a TXT record. Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- .../src/bin/proxmox-fetch-answer.rs | 11 +- .../src/fetch_plugins/http.rs | 185 ++++++++++++++++++ .../src/fetch_plugins/mod.rs | 1 + unconfigured.sh | 9 + 4 files changed, 203 insertions(+), 3 deletions(-) create mode 100644 proxmox-auto-installer/src/fetch_plugins/http.rs diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs index 9e89a3c..f5aeb92 100644 --- a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs +++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs @@ -1,6 +1,9 @@ use anyhow::{anyhow, Error, Result}; use log::{error, info, LevelFilter}; -use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger}; +use proxmox_auto_installer::{ + fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition}, + log::AutoInstLogger, +}; use std::io::Write; use std::process::{Command, ExitCode, Stdio}; @@ -18,8 +21,10 @@ fn fetch_answer() -> Result<String> { Ok(answer) => return Ok(answer), Err(err) => info!("Fetching answer file from partition failed: {err}"), } - // TODO: add more options to get an answer file, e.g. download from url where url could be - // fetched via txt records on predefined subdomain, kernel param, dhcp option, ... + match FetchFromHTTP::get_answer() { + Ok(answer) => return Ok(answer), + Err(err) => info!("Fetching answer file via HTTP failed: {err}"), + } Err(Error::msg("Could not find any answer file!")) } diff --git a/proxmox-auto-installer/src/fetch_plugins/http.rs b/proxmox-auto-installer/src/fetch_plugins/http.rs new file mode 100644 index 0000000..b9fdca7 --- /dev/null +++ b/proxmox-auto-installer/src/fetch_plugins/http.rs @@ -0,0 +1,185 @@ +use anyhow::{bail, Error, Result}; +use log::info; +use std::{ + fs::{self, read_to_string}, + path::Path, + process::Command, +}; + +use crate::fetch_plugins::utils::{post, sysinfo}; + +use super::utils; + +static CERT_FINGERPRINT_FILE: &str = "cert_fingerprint.txt"; +static ANSWER_SUBDOMAIN: &str = "proxmoxinst"; +static ANSWER_SUBDOMAIN_FP: &str = "proxmoxinst-fp"; + +// It is possible to set custom DHPC options. Option numbers 224 to 254 [0]. +// To use them with dhclient, we need to configure it to request them and what they should be +// called. +// +// e.g. /etc/dhcp/dhclient.conf: +// ``` +// option proxmoxinst-url code 250 = text; +// option proxmoxinst-fp code 251 = text; +// also request proxmoxinst-url, proxmoxinst-fp; +// ``` +// +// The results will end up in the /var/lib/dhcp/dhclient.leases file from where we can fetch them +// +// [0] https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml +static DHCP_URL_OPTION: &str = "proxmoxinst-url"; +static DHCP_FP_OPTION: &str = "proxmoxinst-fp"; +static DHCP_LEASE_FILE: &str = "/var/lib/dhcp/dhclient.leases"; + +pub struct FetchFromHTTP; + +impl FetchFromHTTP { + /// Will try to fetch the answer.toml by sending a HTTP POST request. The URL can be configured + /// either via DHCP or DNS. + /// DHCP options are checked first. The SSL certificate need to be either trusted by the root + /// certs or a SHA256 fingerprint needs to be provided. The SHA256 SSL fingerprint can either + /// be placed in a `cert_fingerprint.txt` file in the `proxmoxinst` partition, as DHCP option, + /// or as DNS TXT record. If provided, the `cert_fingerprint.txt` file has preference. + pub fn get_answer() -> Result<String> { + info!("Checking for certificate fingerprint in file."); + let mut fingerprint: Option<String> = match Self::get_cert_fingerprint_from_file() { + Ok(fp) => Some(fp), + Err(err) => { + info!("{err}"); + None + } + }; + + let answer_url: String; + + (answer_url, fingerprint) = match Self::fetch_dhcp(fingerprint.clone()) { + Ok((url, fp)) => (url, fp), + Err(err) => { + info!("{err}"); + Self::fetch_dns(fingerprint.clone())? + } + }; + + info!("Gathering system information."); + let payload = sysinfo::get_sysinfo(false)?; + info!("Sending POST request to '{answer_url}'."); + let answer = post::call(answer_url, fingerprint.as_deref(), payload)?; + Ok(answer) + } + + /// Reads certificate fingerprint from file + pub fn get_cert_fingerprint_from_file() -> Result<String> { + let mount_path = utils::mount_proxmoxinst_part()?; + let cert_path = Path::new(mount_path.as_str()).join(CERT_FINGERPRINT_FILE); + match cert_path.try_exists() { + Ok(true) => { + info!("Found certifacte fingerprint file."); + Ok(fs::read_to_string(cert_path)?.trim().into()) + } + _ => Err(Error::msg(format!( + "could not find cert fingerprint file expected at: {}", + cert_path.display() + ))), + } + } + + /// Fetches search domain from resolv.conf file + fn get_search_domain() -> Result<String> { + info!("Retrieving default search domain."); + for line in read_to_string("/etc/resolv.conf")?.lines() { + if let Some((key, value)) = line.split_once(' ') { + if key == "search" { + return Ok(value.trim().into()); + } + } + } + Err(Error::msg("Could not find search domain in resolv.conf.")) + } + + /// Runs a TXT DNS query on the domain provided + fn query_txt_record(query: String) -> Result<String> { + info!("Querying TXT record for '{query}'"); + let url: String; + match Command::new("dig") + .args(["txt", "+short"]) + .arg(&query) + .output() + { + Ok(output) => { + if output.status.success() { + url = String::from_utf8(output.stdout)? + .replace('"', "") + .trim() + .into(); + if url.is_empty() { + bail!("Got empty response."); + } + } else { + bail!( + "Error querying DNS record '{query}' : {}", + String::from_utf8(output.stderr)? + ); + } + } + Err(err) => bail!("Error querying DNS record '{query}': {err}"), + } + info!("Found: '{url}'"); + Ok(url) + } + + /// Tries to fetch answer URL and SSL fingerprint info from DNS + fn fetch_dns(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> { + let search_domain = Self::get_search_domain()?; + + let answer_url = match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN}.{search_domain}")) + { + Ok(url) => url, + Err(err) => bail!("{err}"), + }; + + if fingerprint.is_none() { + fingerprint = + match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN_FP}.{search_domain}")) { + Ok(fp) => Some(fp), + Err(err) => { + info!("{err}"); + None + } + }; + } + Ok((answer_url, fingerprint)) + } + + /// Tries to fetch answer URL and SSL fingerprint info from DHCP options + fn fetch_dhcp(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> { + let leases = fs::read_to_string(DHCP_LEASE_FILE)?; + + let mut answer_url: Option<String> = None; + + let url_match = format!("option {DHCP_URL_OPTION}"); + let fp_match = format!("option {DHCP_FP_OPTION}"); + + for line in leases.lines() { + if answer_url.is_none() && line.trim().starts_with(url_match.as_str()) { + answer_url = Self::strip_dhcp_option(line.split(' ').nth_back(0)); + } + if fingerprint.is_none() && line.trim().starts_with(fp_match.as_str()) { + fingerprint = Self::strip_dhcp_option(line.split(' ').nth_back(0)); + } + } + + let answer_url = match answer_url { + None => bail!("No DHCP option found for fetch URL."), + Some(url) => url, + }; + + Ok((answer_url, fingerprint)) + } + + /// Clean DHCP option string + fn strip_dhcp_option(value: Option<&str>) -> Option<String> { + // value is expected to be in format: "value"; + value.map(|value| String::from(&value[1..value.len() - 2])) + } +} diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs index 6f1e8a2..354fa7e 100644 --- a/proxmox-auto-installer/src/fetch_plugins/mod.rs +++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs @@ -1,2 +1,3 @@ +pub mod http; pub mod partition; pub mod utils; diff --git a/unconfigured.sh b/unconfigured.sh index f668be4..fd0fb53 100755 --- a/unconfigured.sh +++ b/unconfigured.sh @@ -205,6 +205,15 @@ if [ $proxdebug -ne 0 ]; then debugsh || true fi +# add custom DHCP options for auto installer +if [ $proxauto -ne 0 ]; then + cat >> /etc/dhcp/dhclient.conf <<EOF +option proxmoxinst-url code 250 = text; +option proxmoxinst-fp code 251 = text; +also request proxmoxinst-url, proxmoxinst-fp; +EOF +fi + # try to get ip config with dhcp echo -n "Attempting to get DHCP leases... " dhclient -v -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [pve-devel] [PATCH v2 22/22] control: update build depends for auto installer 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (20 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 21/22] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer @ 2024-02-21 11:08 ` Aaron Lauterer 2024-02-21 13:39 ` [pve-devel] [PATCH v2 00/22] add automated/unattended installation Christoph Heiss 2024-02-23 10:19 ` Friedrich Weber 23 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-21 11:08 UTC (permalink / raw) To: pve-devel Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> --- debian/control | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/debian/control b/debian/control index 9057f59..72a5657 100644 --- a/debian/control +++ b/debian/control @@ -8,10 +8,20 @@ Build-Depends: cargo:native, libgtk3-perl, libpve-common-perl, librsvg2-bin, + librust-anyhow-dev, + librust-clap-dev, librust-cursive+termion-backend-dev (>= 0.20.0), + librust-glob-dev, + librust-hex-dev, + librust-native-tls-dev, librust-regex-1+default-dev (>= 1.7~~), + librust-rustls-dev, + librust-rustls-native-certs-dev, librust-serde-1+default-dev, librust-serde-json-1+default-dev, + librust-sha2-dev, + librust-ureq-dev, + librust-toml-dev, libtest-mockmodule-perl, perl, rustc:native, -- 2.39.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 00/22] add automated/unattended installation 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (21 preceding siblings ...) 2024-02-21 11:08 ` [pve-devel] [PATCH v2 22/22] control: update build depends for auto installer Aaron Lauterer @ 2024-02-21 13:39 ` Christoph Heiss 2024-02-23 10:19 ` Friedrich Weber 23 siblings, 0 replies; 29+ messages in thread From: Christoph Heiss @ 2024-02-21 13:39 UTC (permalink / raw) To: Aaron Lauterer; +Cc: Proxmox VE development discussion On Wed, Feb 21, 2024 at 12:07:43PM +0100, Aaron Lauterer wrote: > This patch series adds the possibility to do an automated / unattended > installation of Proxmox VE. > [..] Before anyone else stumbles into this while testing/reviewing, I've also just talked with Aaron over this: Basically, the build is a bit racy and can fail to due a missing prerequisite in the Makefile. The proper fix boils down to: diff --git a/Makefile b/Makefile index dec5b7b..89b4cbb 100644 --- a/Makefile +++ b/Makefile @@ -99,7 +99,7 @@ VARLIBDIR=$(DESTDIR)/var/lib/proxmox-installer HTMLDIR=$(VARLIBDIR)/html/common .PHONY: install -install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer +install: $(INSTALLER_SOURCES) $(COMPILED_BINS) $(MAKE) -C banner install $(MAKE) -C Proxmox install install -D -m 644 interfaces $(DESTDIR)/etc/network/interfaces @@ -118,7 +118,11 @@ install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer $(COMPILED_BINS): cargo-build .PHONY: cargo-build cargo-build: - $(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer $(CARGO_BUILD_ARGS) + $(CARGO) build \ + --package proxmox-tui-installer --bin proxmox-tui-installer \ + --package proxmox-auto-installer --bin proxmox-tui-installer \ + --bin proxmox-autoinst-helper --bin proxmox-fetch-answer \ + $(CARGO_BUILD_ARGS) %-banner.png: %-banner.svg rsvg-convert -o $@ $< ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 00/22] add automated/unattended installation 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer ` (22 preceding siblings ...) 2024-02-21 13:39 ` [pve-devel] [PATCH v2 00/22] add automated/unattended installation Christoph Heiss @ 2024-02-23 10:19 ` Friedrich Weber 2024-02-23 11:37 ` Aaron Lauterer 23 siblings, 1 reply; 29+ messages in thread From: Friedrich Weber @ 2024-02-23 10:19 UTC (permalink / raw) To: Proxmox VE development discussion, Aaron Lauterer On 21/02/2024 12:07, Aaron Lauterer wrote: > This patch series adds the possibility to do an automated / unattended > installation of Proxmox VE. Gave this a quick spin installing some virtual PVE hosts with a simple static IP + ext4 setup. Generated an ISO from my `answer.toml` with: $ mkisofs -o /var/lib/vz/template/iso/test.iso -iso-level 3 -V proxmoxinst answer.toml Attached it as a second CD/DVD drive to the VM, works quite well! :) One thing I noticed: At first I forgot to set any `filter.` option in the [network] section. As a result I got an "Installation aborted" after "Setting network configuration", and nothing more in /tmp/auto_installer.log. It makes sense that the installer can't apply the settings without a device filter, but it would be good to get an error message. AFAICT the same thing happens if there is no filter in [disks]. I hope to do some more tests next week, but wanted to report the above already today. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [pve-devel] [PATCH v2 00/22] add automated/unattended installation 2024-02-23 10:19 ` Friedrich Weber @ 2024-02-23 11:37 ` Aaron Lauterer 0 siblings, 0 replies; 29+ messages in thread From: Aaron Lauterer @ 2024-02-23 11:37 UTC (permalink / raw) To: Friedrich Weber, Proxmox VE development discussion On 2/23/24 11:19, Friedrich Weber wrote: > On 21/02/2024 12:07, Aaron Lauterer wrote: >> This patch series adds the possibility to do an automated / unattended >> installation of Proxmox VE. > > Gave this a quick spin installing some virtual PVE hosts with a simple > static IP + ext4 setup. Generated an ISO from my `answer.toml` with: > > $ mkisofs -o /var/lib/vz/template/iso/test.iso -iso-level 3 -V > proxmoxinst answer.toml > > Attached it as a second CD/DVD drive to the VM, works quite well! :) > > One thing I noticed: At first I forgot to set any `filter.` option in > the [network] section. As a result I got an "Installation aborted" after > "Setting network configuration", and nothing more in > /tmp/auto_installer.log. It makes sense that the installer can't apply > the settings without a device filter, but it would be good to get an > error message. AFAICT the same thing happens if there is no filter in > [disks]. Thanks! I will see how these checks can be made earlier and in the helper tool as well when validating an answer file. > > I hope to do some more tests next week, but wanted to report the above > already today. ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2024-02-23 11:37 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-02-21 11:07 [pve-devel] [PATCH v2 00/22] add automated/unattended installation Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 01/22] tui: common: move InstallConfig struct to common crate Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 02/22] common: make InstallZfsOption members public Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 03/22] common: tui: use BTreeMap for predictable ordering Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 04/22] low-level: add dump-udev command Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 05/22] add auto-installer crate Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 06/22] auto-installer: add dependencies Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 07/22] auto-installer: add answer file definition Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 08/22] auto-installer: add struct to hold udev info Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 09/22] auto-installer: add utils Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 10/22] auto-installer: add simple logging Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 11/22] auto-installer: add tests for answer file parsing Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 12/22] auto-installer: add auto-installer binary Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 13/22] auto-installer: add fetch answer binary Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 14/22] unconfigured: add proxauto as option to start auto installer Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 15/22] auto-installer: use glob crate for pattern matching Aaron Lauterer 2024-02-21 11:07 ` [pve-devel] [PATCH v2 16/22] auto-installer: utils: make get_udev_index functions public Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 17/22] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 18/22] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer 2024-02-21 14:09 ` Christoph Heiss 2024-02-21 16:07 ` Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 19/22] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 20/22] auto-installer: fetch: add http post utility module Aaron Lauterer 2024-02-21 12:21 ` Christoph Heiss 2024-02-21 11:08 ` [pve-devel] [PATCH v2 21/22] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer 2024-02-21 11:08 ` [pve-devel] [PATCH v2 22/22] control: update build depends for auto installer Aaron Lauterer 2024-02-21 13:39 ` [pve-devel] [PATCH v2 00/22] add automated/unattended installation Christoph Heiss 2024-02-23 10:19 ` Friedrich Weber 2024-02-23 11:37 ` Aaron Lauterer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox