* [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation
@ 2024-04-04 14:48 Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 01/30] tui: common: move InstallConfig struct to common crate Aaron Lauterer
` (31 more replies)
0 siblings, 32 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
This patch series adds the possibility to do an automated / unattended
installation of Proxmox VE.
The overall idea is that we will have a dedicated ISO for the unattended
installation. It should be configured in such a way that it will start
the installation without any user interaction. Therefore, the GRUB
config should automatically start it (after a timeout).
The information for the installer that is usually gathered interactively
from the user is provided via an `answer.toml` file.
The answer file allows to select disks and the network card via filters.
The installer also allows to run custom commands pre and post
installation. This should give users plenty of possibilities to either
further customize/prepare the installation or integrate it into a larger
automated installation setup.
For example, one could issue HTTP requests to signal the status and
progress of the installation.
When the installer is called with 'proxauto' in the kernel cmdline, the
'proxmox-fetch-answer' binary is called. It tries to find the answer
file and once found, will start the 'proxmox-auto-installer' binary and
pass the contents to it via stdin.
The auto-installer then parses the answer file and determines what
parameters need to be passed to the low-level installer. For example,
which disks and NIC to use, network IP settings and so forth.
The current status reporting of the actual installation is kept rather
simple.
Both binaries log into the tmp directory.
There is a third binary, the 'proxmox-autoinst-helper'. It provides a
few subcommands, from the help:
answer Validate if an answer file is formatted correctly
device-match Test which devices the given filter matches against
device-info Show device information that can be used for filters
identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file
The fetch-answer binary is trying to get an answer file. It does so by
first searching for a partition/FS labeled `proxmoxinst`, or all upper
case, and an `answer.toml` in there. This could be provided by another
USB flash drive.
If that is not successful, the next step is to send an HTTP POST request
to a URL to get the TOML contents in return. A POST request was chosen
because we also send information to identify the host in JSON format.
The question then is, where to get that URL from. Right now, there are
two options implemented. The first is looking for a custom DHCP option
and the second is querying for a TXT record in the `proxmoxinst`
subdomain of the search domain.
It is possible to provide a SHA256 fingerprint of the SSL cert used by
the answer server. The safest option is to place a
`cert_fingerprint.txt` file in the same `proxmoxinst` partition as where
you alternatively would place the `answer.toml`.
If that is not found, then it can be provided by a second custom DHCP
option or placed as TXT record in the subdomain `proxmoxinst-fp`.
This patch series now also separates the 3 binaries into their own
crate. The 'proxmox-fetch-answer' to keep the OpenSSL dependency as
localized as possible, and the 'proxmox-autoinst-helper' to make it easy
to compile just that binary.
The new `proxmox-chroot` utility helps to prepare everything to chroot
into a fresh installation and clean it up once done.
This will be useful in the post commands when further customizing the
installation.
Other plans / ideas for the future:
* add option to define remote SSH access (password and,or public key).
This could make remote debugging in case of problems easier
Regarding the patch series itself:
01-03 are needed to move some code into the common crate and
make structs/functions already in the common crate accessible.
I did split up the individual parts of the auto installer into their own
patches as much as possible, and (hopefully) in the order they depend on
each other.
Patches after the `unconfigured` one (16), switch the pattern matching
to the glob crate, add the helper tool and the fetching via HTTP.
Patch 26 factors our the binaries into their own crates.
Patches 27-30 are for the 'proxmox-chroot' utility and preparations for
it to work.
Areas that can be improved/extended:
* Testing possibility integrated in the Makefile
I did test it with all 3 installers, PVE, PMG and PBS and it worked.
WIP: Documentation. A first draft is available in the inernal wiki, as
we will most likely keep it in wiki format since it applies for all 3
products, if we provide ISOs for it.
since v3:
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Changes since V3:
* implement most suggested code changes. Thx @Christoph for reviewing it
* reordered patches a little bit. While testing individual changes I
realized that some patches needed reordering and rebasing
* improved error handling of pre- and post-commands. Errors will now be
logged & printed.
Changes since v2:
* don't use 'dmidecode' but check in the source locations directly for
identifiers
* fixed makefile to get builds working every time
* reworked filesystem and raid level layout in the answer file
definition
* factor out binaries into their own crates
* add 'proxmox-chroot' helper tool
Changes since v1:
* incorporated feedback (thx @cheiss)
* the proxmox-installer-filter tool got renamed and extended to the
proxmox-autoinst-helper
* added HTTP fetch functionality
* moved the fetch_plugins/utils.rs into its own subdirectory as it grew
quite a bit
* added builddeps
Aaron Lauterer (30):
tui: common: move InstallConfig struct to common crate
common: make InstallZfsOption members public
common: tui: use BTreeMap for predictable ordering
common: utils: add deserializer for CidrAddress
common: options: add Deserialize trait
low-level: add dump-udev command
add auto-installer crate
auto-installer: add dependencies
auto-installer: add answer file definition
auto-installer: add struct to hold udev info
auto-installer: add utils
auto-installer: add simple logging
auto-installer: add tests for answer file parsing
auto-installer: add auto-installer binary
auto-installer: add fetch answer binary
unconfigured: add proxauto as option to start auto installer
auto-installer: use glob crate for pattern matching
auto-installer: utils: make get_udev_index functions public
auto-installer: add proxmox-autoinst-helper tool
common: add Display trait to ProxmoxProduct
auto-installer: fetch: add gathering of system identifiers and
restructure code
auto-installer: helper: add subcommand to view indentifiers
auto-installer: fetch: add http post utility module
auto-installer: fetch: add http plugin to fetch answer
control: update build depends for auto installer
auto installer: factor out fetch-answer and autoinst-helper
low-level: write low level config to /tmp
common: add deserializer for FsType
common: skip target_hd when deserializing InstallConfig
add proxmox-chroot utility
Cargo.toml | 4 +
Makefile | 19 +-
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 +++
debian/control | 10 +
proxmox-auto-installer/Cargo.toml | 20 +
proxmox-auto-installer/src/answer.rs | 249 ++++++++++
.../src/bin/proxmox-auto-installer.rs | 195 ++++++++
proxmox-auto-installer/src/lib.rs | 5 +
proxmox-auto-installer/src/log.rs | 38 ++
proxmox-auto-installer/src/sysinfo.rs | 81 ++++
proxmox-auto-installer/src/udevinfo.rs | 9 +
proxmox-auto-installer/src/utils.rs | 437 ++++++++++++++++++
proxmox-auto-installer/tests/parse-answer.rs | 106 +++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 ++
.../resources/parse_answer/disk_match.toml | 17 +
.../parse_answer/disk_match_all.json | 26 ++
.../parse_answer/disk_match_all.toml | 17 +
.../parse_answer/disk_match_any.json | 33 ++
.../parse_answer/disk_match_any.toml | 17 +
.../tests/resources/parse_answer/minimal.json | 17 +
.../tests/resources/parse_answer/minimal.toml | 14 +
.../resources/parse_answer/nic_matching.json | 17 +
.../resources/parse_answer/nic_matching.toml | 19 +
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +
.../resources/parse_answer/specific_nic.toml | 19 +
.../tests/resources/parse_answer/zfs.json | 27 ++
.../tests/resources/parse_answer/zfs.toml | 20 +
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
proxmox-autoinst-helper/Cargo.toml | 21 +
proxmox-autoinst-helper/src/main.rs | 337 ++++++++++++++
proxmox-chroot/Cargo.toml | 16 +
proxmox-chroot/src/main.rs | 356 ++++++++++++++
proxmox-fetch-answer/Cargo.toml | 22 +
.../src/fetch_plugins/http.rs | 191 ++++++++
proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 +
.../src/fetch_plugins/partition.rs | 32 ++
.../src/fetch_plugins/utils/mod.rs | 85 ++++
.../src/fetch_plugins/utils/post.rs | 94 ++++
proxmox-fetch-answer/src/main.rs | 76 +++
proxmox-installer-common/Cargo.toml | 1 +
proxmox-installer-common/src/options.rs | 21 +-
proxmox-installer-common/src/setup.rs | 141 +++++-
proxmox-installer-common/src/utils.rs | 11 +
proxmox-low-level-installer | 14 +
proxmox-tui-installer/src/options.rs | 4 +-
proxmox-tui-installer/src/setup.rs | 100 +---
.../src/views/install_progress.rs | 4 +-
unconfigured.sh | 17 +
53 files changed, 2953 insertions(+), 118 deletions(-)
create mode 100644 Proxmox/Sys/Udev.pm
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
create mode 100644 proxmox-auto-installer/src/lib.rs
create mode 100644 proxmox-auto-installer/src/log.rs
create mode 100644 proxmox-auto-installer/src/sysinfo.rs
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
create mode 100644 proxmox-auto-installer/src/utils.rs
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
create mode 100644 proxmox-autoinst-helper/Cargo.toml
create mode 100644 proxmox-autoinst-helper/src/main.rs
create mode 100644 proxmox-chroot/Cargo.toml
create mode 100644 proxmox-chroot/src/main.rs
create mode 100644 proxmox-fetch-answer/Cargo.toml
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/http.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
create mode 100644 proxmox-fetch-answer/src/main.rs
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 01/30] tui: common: move InstallConfig struct to common crate
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 02/30] common: make InstallZfsOption members public Aaron Lauterer
` (30 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
It describes the data structure expected by the low-level-installer.
We do this so we can use it in more than the TUI installer, for example
the planned auto installer.
Make the members public so we can easily implement a custom From method
for each dependent crate.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 86 +++++++++++++++-
proxmox-tui-installer/src/setup.rs | 98 +------------------
.../src/views/install_progress.rs | 4 +-
3 files changed, 90 insertions(+), 98 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 472e1f2..03beb77 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -12,7 +12,10 @@ use std::{
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
use crate::{
- options::{Disk, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption},
+ options::{
+ BtrfsRaidLevel, Disk, FsType, ZfsBootdiskOptions, ZfsChecksumOption, ZfsCompressOption,
+ ZfsRaidLevel,
+ },
utils::CidrAddress,
};
@@ -387,3 +390,84 @@ pub fn spawn_low_level_installer(test_mode: bool) -> io::Result<process::Child>
.stdout(Stdio::piped())
.spawn()
}
+
+/// See Proxmox::Install::Config
+#[derive(Serialize)]
+pub struct InstallConfig {
+ pub autoreboot: usize,
+
+ #[serde(serialize_with = "serialize_fstype")]
+ pub filesys: FsType,
+ pub hdsize: f64,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub swapsize: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub maxroot: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub minfree: Option<f64>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub maxvz: Option<f64>,
+
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub zfs_opts: Option<InstallZfsOption>,
+
+ #[serde(
+ serialize_with = "serialize_disk_opt",
+ skip_serializing_if = "Option::is_none"
+ )]
+ pub target_hd: Option<Disk>,
+ #[serde(skip_serializing_if = "HashMap::is_empty")]
+ pub disk_selection: HashMap<String, String>,
+
+ pub country: String,
+ pub timezone: String,
+ pub keymap: String,
+
+ pub password: String,
+ pub mailto: String,
+
+ pub mngmt_nic: String,
+
+ pub hostname: String,
+ pub domain: String,
+ #[serde(serialize_with = "serialize_as_display")]
+ pub cidr: CidrAddress,
+ pub gateway: IpAddr,
+ pub dns: IpAddr,
+}
+
+fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error>
+where
+ S: Serializer,
+{
+ if let Some(disk) = value {
+ serializer.serialize_str(&disk.path)
+ } else {
+ serializer.serialize_none()
+ }
+}
+
+fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error>
+where
+ S: Serializer,
+{
+ use FsType::*;
+ let value = match value {
+ // proxinstall::$fssetup
+ Ext4 => "ext4",
+ Xfs => "xfs",
+ // proxinstall::get_zfs_raid_setup()
+ Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)",
+ Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)",
+ Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)",
+ Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)",
+ Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)",
+ Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)",
+ // proxinstall::get_btrfs_raid_setup()
+ Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)",
+ Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)",
+ Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)",
+ };
+
+ serializer.collect_str(value)
+}
diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs
index 79421d7..e816c12 100644
--- a/proxmox-tui-installer/src/setup.rs
+++ b/proxmox-tui-installer/src/setup.rs
@@ -1,59 +1,11 @@
-use std::{collections::HashMap, fmt, net::IpAddr};
-
-use serde::{Serialize, Serializer};
+use std::collections::HashMap;
use crate::options::InstallerOptions;
use proxmox_installer_common::{
- options::{AdvancedBootdiskOptions, BtrfsRaidLevel, Disk, FsType, ZfsRaidLevel},
- setup::InstallZfsOption,
- utils::CidrAddress,
+ options::AdvancedBootdiskOptions,
+ setup::InstallConfig,
};
-/// See Proxmox::Install::Config
-#[derive(Serialize)]
-pub struct InstallConfig {
- autoreboot: usize,
-
- #[serde(serialize_with = "serialize_fstype")]
- filesys: FsType,
- hdsize: f64,
- #[serde(skip_serializing_if = "Option::is_none")]
- swapsize: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- maxroot: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- minfree: Option<f64>,
- #[serde(skip_serializing_if = "Option::is_none")]
- maxvz: Option<f64>,
-
- #[serde(skip_serializing_if = "Option::is_none")]
- zfs_opts: Option<InstallZfsOption>,
-
- #[serde(
- serialize_with = "serialize_disk_opt",
- skip_serializing_if = "Option::is_none"
- )]
- target_hd: Option<Disk>,
- #[serde(skip_serializing_if = "HashMap::is_empty")]
- disk_selection: HashMap<String, String>,
-
- country: String,
- timezone: String,
- keymap: String,
-
- password: String,
- mailto: String,
-
- mngmt_nic: String,
-
- hostname: String,
- domain: String,
- #[serde(serialize_with = "serialize_as_display")]
- cidr: CidrAddress,
- gateway: IpAddr,
- dns: IpAddr,
-}
-
impl From<InstallerOptions> for InstallConfig {
fn from(options: InstallerOptions) -> Self {
let mut config = Self {
@@ -121,47 +73,3 @@ impl From<InstallerOptions> for InstallConfig {
config
}
}
-
-fn serialize_disk_opt<S>(value: &Option<Disk>, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
-{
- if let Some(disk) = value {
- serializer.serialize_str(&disk.path)
- } else {
- serializer.serialize_none()
- }
-}
-
-fn serialize_fstype<S>(value: &FsType, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
-{
- use FsType::*;
- let value = match value {
- // proxinstall::$fssetup
- Ext4 => "ext4",
- Xfs => "xfs",
- // proxinstall::get_zfs_raid_setup()
- Zfs(ZfsRaidLevel::Raid0) => "zfs (RAID0)",
- Zfs(ZfsRaidLevel::Raid1) => "zfs (RAID1)",
- Zfs(ZfsRaidLevel::Raid10) => "zfs (RAID10)",
- Zfs(ZfsRaidLevel::RaidZ) => "zfs (RAIDZ-1)",
- Zfs(ZfsRaidLevel::RaidZ2) => "zfs (RAIDZ-2)",
- Zfs(ZfsRaidLevel::RaidZ3) => "zfs (RAIDZ-3)",
- // proxinstall::get_btrfs_raid_setup()
- Btrfs(BtrfsRaidLevel::Raid0) => "btrfs (RAID0)",
- Btrfs(BtrfsRaidLevel::Raid1) => "btrfs (RAID1)",
- Btrfs(BtrfsRaidLevel::Raid10) => "btrfs (RAID10)",
- };
-
- serializer.collect_str(value)
-}
-
-fn serialize_as_display<S, T>(value: &T, serializer: S) -> Result<S::Ok, S::Error>
-where
- S: Serializer,
- T: fmt::Display,
-{
- serializer.collect_str(value)
-}
diff --git a/proxmox-tui-installer/src/views/install_progress.rs b/proxmox-tui-installer/src/views/install_progress.rs
index c2c9ddf..56356ad 100644
--- a/proxmox-tui-installer/src/views/install_progress.rs
+++ b/proxmox-tui-installer/src/views/install_progress.rs
@@ -13,8 +13,8 @@ use std::{
time::Duration,
};
-use crate::{abort_install_button, prompt_dialog, setup::InstallConfig, InstallerState};
-use proxmox_installer_common::setup::spawn_low_level_installer;
+use crate::{abort_install_button, prompt_dialog, InstallerState};
+use proxmox_installer_common::setup::{InstallConfig, spawn_low_level_installer};
pub struct InstallProgressView {
view: PaddedView<LinearLayout>,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 02/30] common: make InstallZfsOption members public
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 01/30] tui: common: move InstallConfig struct to common crate Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 03/30] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
` (29 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
as they will be used directly by the auto installer
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 03beb77..1bc4cb7 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -147,13 +147,13 @@ pub fn installer_setup(in_test_mode: bool) -> Result<(SetupInfo, LocaleInfo, Run
#[derive(Serialize)]
pub struct InstallZfsOption {
- ashift: usize,
+ pub ashift: usize,
#[serde(serialize_with = "serialize_as_display")]
- compress: ZfsCompressOption,
+ pub compress: ZfsCompressOption,
#[serde(serialize_with = "serialize_as_display")]
- checksum: ZfsChecksumOption,
- copies: usize,
- arc_max: usize,
+ pub checksum: ZfsChecksumOption,
+ pub copies: usize,
+ pub arc_max: usize,
}
impl From<ZfsBootdiskOptions> for InstallZfsOption {
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 03/30] common: tui: use BTreeMap for predictable ordering
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 01/30] tui: common: move InstallConfig struct to common crate Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 02/30] common: make InstallZfsOption members public Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 04/30] common: utils: add deserializer for CidrAddress Aaron Lauterer
` (28 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
necessary for the disk selection and network interfaces maps to have
tests with results that can be compared without much additional effort.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 8 ++++----
proxmox-tui-installer/src/options.rs | 4 ++--
proxmox-tui-installer/src/setup.rs | 4 ++--
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 1bc4cb7..8432a2c 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -1,6 +1,6 @@
use std::{
cmp,
- collections::HashMap,
+ collections::{BTreeMap, HashMap},
fmt,
fs::File,
io::{self, BufReader},
@@ -297,7 +297,7 @@ pub struct NetworkInfo {
/// Maps devices to their configuration, if it has a usable configuration.
/// (Contains no entries for devices with only link-local addresses.)
#[serde(default)]
- pub interfaces: HashMap<String, Interface>,
+ pub interfaces: BTreeMap<String, Interface>,
/// The hostname of this machine, if set by the DHCP server.
pub hostname: Option<String>,
@@ -416,8 +416,8 @@ pub struct InstallConfig {
skip_serializing_if = "Option::is_none"
)]
pub target_hd: Option<Disk>,
- #[serde(skip_serializing_if = "HashMap::is_empty")]
- pub disk_selection: HashMap<String, String>,
+ #[serde(skip_serializing_if = "BTreeMap::is_empty")]
+ pub disk_selection: BTreeMap<String, String>,
pub country: String,
pub timezone: String,
diff --git a/proxmox-tui-installer/src/options.rs b/proxmox-tui-installer/src/options.rs
index 094a430..73fbf2a 100644
--- a/proxmox-tui-installer/src/options.rs
+++ b/proxmox-tui-installer/src/options.rs
@@ -76,7 +76,7 @@ mod tests {
utils::{CidrAddress, Fqdn},
};
use std::net::{IpAddr, Ipv4Addr};
- use std::{collections::HashMap, path::PathBuf};
+ use std::{collections::BTreeMap, path::PathBuf};
fn dummy_setup_info() -> SetupInfo {
SetupInfo {
@@ -99,7 +99,7 @@ mod tests {
fn network_options_from_setup_network_info() {
let setup = dummy_setup_info();
- let mut interfaces = HashMap::new();
+ let mut interfaces = BTreeMap::new();
interfaces.insert(
"eth0".to_owned(),
Interface {
diff --git a/proxmox-tui-installer/src/setup.rs b/proxmox-tui-installer/src/setup.rs
index e816c12..248f86e 100644
--- a/proxmox-tui-installer/src/setup.rs
+++ b/proxmox-tui-installer/src/setup.rs
@@ -1,4 +1,4 @@
-use std::collections::HashMap;
+use std::collections::BTreeMap;
use crate::options::InstallerOptions;
use proxmox_installer_common::{
@@ -19,7 +19,7 @@ impl From<InstallerOptions> for InstallConfig {
maxvz: None,
zfs_opts: None,
target_hd: None,
- disk_selection: HashMap::new(),
+ disk_selection: BTreeMap::new(),
country: options.timezone.country,
timezone: options.timezone.timezone,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 04/30] common: utils: add deserializer for CidrAddress
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (2 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 03/30] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 05/30] common: options: add Deserialize trait Aaron Lauterer
` (27 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
so it can be deserialized from a string
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/utils.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/proxmox-installer-common/src/utils.rs b/proxmox-installer-common/src/utils.rs
index 36b1d53..f6521eb 100644
--- a/proxmox-installer-common/src/utils.rs
+++ b/proxmox-installer-common/src/utils.rs
@@ -103,6 +103,17 @@ impl fmt::Display for CidrAddress {
}
}
+impl<'de> Deserialize<'de> for CidrAddress {
+ fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
+ where
+ D: serde::Deserializer<'de>,
+ {
+ let s: String = Deserialize::deserialize(deserializer)?;
+ s.parse()
+ .map_err(|_| serde::de::Error::custom("invalid CIDR"))
+ }
+}
+
fn mask_limit(addr: &IpAddr) -> usize {
if addr.is_ipv4() {
32
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 05/30] common: options: add Deserialize trait
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (3 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 04/30] common: utils: add deserializer for CidrAddress Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 06/30] low-level: add dump-udev command Aaron Lauterer
` (26 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
For the Enums that will be used to deserialize an answer file.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/options.rs | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/proxmox-installer-common/src/options.rs b/proxmox-installer-common/src/options.rs
index 1aa8f65..1efac66 100644
--- a/proxmox-installer-common/src/options.rs
+++ b/proxmox-installer-common/src/options.rs
@@ -1,12 +1,14 @@
use std::net::{IpAddr, Ipv4Addr};
use std::{cmp, fmt};
+use serde::Deserialize;
use crate::setup::{
LocaleInfo, NetworkInfo, ProductConfig, ProxmoxProduct, RuntimeInfo, SetupInfo,
};
use crate::utils::{CidrAddress, Fqdn};
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum BtrfsRaidLevel {
Raid0,
Raid1,
@@ -24,7 +26,8 @@ impl fmt::Display for BtrfsRaidLevel {
}
}
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum ZfsRaidLevel {
Raid0,
Raid1,
@@ -112,7 +115,8 @@ impl BtrfsBootdiskOptions {
}
}
-#[derive(Copy, Clone, Debug, Default, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Default, Deserialize, Eq, PartialEq)]
+#[serde(rename_all(deserialize = "lowercase"))]
pub enum ZfsCompressOption {
#[default]
On,
@@ -141,7 +145,8 @@ pub const ZFS_COMPRESS_OPTIONS: &[ZfsCompressOption] = {
&[On, Off, Lzjb, Lz4, Zle, Gzip, Zstd]
};
-#[derive(Copy, Clone, Debug, Default, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Default, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "kebab-case")]
pub enum ZfsChecksumOption {
#[default]
On,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 06/30] low-level: add dump-udev command
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (4 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 05/30] common: options: add Deserialize trait Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 07/30] add auto-installer crate Aaron Lauterer
` (25 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Fetches UDEV device properties prepended with 'E:' for NICs and disks.
The result is stored in its own JSON file.
This information is needed to filter for specific devices. Mainly for
the auto-installer for now.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 +++++++++++++++++++++++++++++++++++++
proxmox-low-level-installer | 13 +++++++++
3 files changed, 68 insertions(+)
create mode 100644 Proxmox/Sys/Udev.pm
diff --git a/Proxmox/Makefile b/Proxmox/Makefile
index d49da80..9561d9b 100644
--- a/Proxmox/Makefile
+++ b/Proxmox/Makefile
@@ -16,6 +16,7 @@ PERL_MODULES=\
Sys/Command.pm \
Sys/File.pm \
Sys/Net.pm \
+ Sys/Udev.pm \
UI.pm \
UI/Base.pm \
UI/Gtk3.pm \
diff --git a/Proxmox/Sys/Udev.pm b/Proxmox/Sys/Udev.pm
new file mode 100644
index 0000000..69d674f
--- /dev/null
+++ b/Proxmox/Sys/Udev.pm
@@ -0,0 +1,54 @@
+package Proxmox::Sys::Udev;
+
+use strict;
+use warnings;
+
+use base qw(Exporter);
+our @EXPORT_OK = qw(disk_details);
+
+my $udev_regex = '^E: ([A-Z_]*)=(.*)$';
+
+my sub fetch_udevadm_info {
+ my ($path) = @_;
+
+ my $info = `udevadm info --path $path --query all`;
+ if (!$info) {
+ warn "no details found for device '${path}'\n";
+ next;
+ }
+ my $details = {};
+ for my $line (split('\n', $info)) {
+ if ($line =~ m/$udev_regex/) {
+ $details->{$1} = $2;
+ }
+ }
+ return $details;
+}
+
+# return hash of E: properties returned by udevadm
+sub disk_details {
+ my $result = {};
+ for my $data (@{Proxmox::Sys::Block::get_cached_disks()}) {
+ my $index = @$data[0];
+ my $bd = @$data[5];
+ $result->{$index} = fetch_udevadm_info($bd);
+ }
+ return $result;
+}
+
+
+sub nic_details {
+ my $nic_path = "/sys/class/net";
+ my $result = {};
+
+ my $nics = Proxmox::Sys::Net::get_ip_config()->{ifaces};
+
+ for my $index (keys %$nics) {
+ my $name = $nics->{$index}->{name};
+ my $nic = "${nic_path}/${name}";
+ $result->{$name} = fetch_udevadm_info($nic);
+ }
+ return $result;
+}
+
+1;
diff --git a/proxmox-low-level-installer b/proxmox-low-level-installer
index 2848295..54f689a 100755
--- a/proxmox-low-level-installer
+++ b/proxmox-low-level-installer
@@ -21,6 +21,7 @@ use Time::HiRes qw(usleep);
use Proxmox::Install::ISOEnv;
use Proxmox::Install::RunEnv;
+use Proxmox::Sys::Udev;
use Proxmox::Sys::File qw(file_write_all);
@@ -31,6 +32,7 @@ use Proxmox::UI;
my $commands = {
'dump-env' => 'Dump the current ISO and Hardware environment to base the installer UI on.',
+ 'dump-udev' => 'Dump disk and network device info. Used for the auto installation.',
'start-session' => 'Start an installation session, with command and result transmitted via stdin/out',
'start-session-test' => 'Start an installation TEST session, with command and result transmitted via stdin/out',
'help' => 'Output this usage help.',
@@ -115,6 +117,17 @@ if ($cmd eq 'dump-env') {
my $run_env = Proxmox::Install::RunEnv::query_installation_environment();
my $run_env_serialized = to_json($run_env, {canonical => 1, utf8 => 1}) ."\n";
file_write_all($run_env_file, $run_env_serialized);
+} elsif ($cmd eq 'dump-udev') {
+ my $out_dir = $env->{locations}->{run};
+ make_path($out_dir);
+ die "failed to create output directory '$out_dir'\n" if !-d $out_dir;
+
+ my $output = {};
+ $output->{disks} = Proxmox::Sys::Udev::disk_details();
+ $output->{nics} = Proxmox::Sys::Udev::nic_details();
+
+ my $output_serialized = to_json($output, {canonical => 1, utf8 => 1}) ."\n";
+ file_write_all("$out_dir/run-env-udev.json", $output_serialized);
} elsif ($cmd eq 'start-session') {
Proxmox::UI::init_stdio({}, $env);
read_and_merge_config();
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 07/30] add auto-installer crate
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (5 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 06/30] low-level: add dump-udev command Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 08/30] auto-installer: add dependencies Aaron Lauterer
` (24 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 1 +
Makefile | 1 +
proxmox-auto-installer/Cargo.toml | 10 ++++++++++
proxmox-auto-installer/src/lib.rs | 0
4 files changed, 12 insertions(+)
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index c1bd578..7017ac5 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,6 @@
[workspace]
members = [
+ "proxmox-auto-installer",
"proxmox-installer-common",
"proxmox-tui-installer",
]
diff --git a/Makefile b/Makefile
index af33e90..4f140ca 100644
--- a/Makefile
+++ b/Makefile
@@ -47,6 +47,7 @@ $(BUILDDIR):
interfaces \
proxinstall \
proxmox-low-level-installer \
+ proxmox-auto-installer/ \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
new file mode 100644
index 0000000..75cfb2c
--- /dev/null
+++ b/proxmox-auto-installer/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "proxmox-auto-installer"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+[dependencies]
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
new file mode 100644
index 0000000..e69de29
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 08/30] auto-installer: add dependencies
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (6 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 07/30] add auto-installer crate Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 09/30] auto-installer: add answer file definition Aaron Lauterer
` (23 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 75cfb2c..67218dd 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -8,3 +8,7 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+proxmox-installer-common = { path = "../proxmox-installer-common" }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+toml = "0.7"
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 09/30] auto-installer: add answer file definition
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (7 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 08/30] auto-installer: add dependencies Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 10/30] auto-installer: add struct to hold udev info Aaron Lauterer
` (22 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 1 +
proxmox-auto-installer/src/answer.rs | 248 +++++++++++++++++++++++++++
proxmox-auto-installer/src/lib.rs | 1 +
3 files changed, 250 insertions(+)
create mode 100644 proxmox-auto-installer/src/answer.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 67218dd..80de4fa 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -12,3 +12,4 @@ proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.7"
+enum-iterator = "0.6.0"
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
new file mode 100644
index 0000000..0add95e
--- /dev/null
+++ b/proxmox-auto-installer/src/answer.rs
@@ -0,0 +1,248 @@
+use proxmox_installer_common::{
+ options::{BtrfsRaidLevel, FsType, ZfsChecksumOption, ZfsCompressOption, ZfsRaidLevel},
+ utils::{CidrAddress, Fqdn},
+};
+use serde::{Deserialize, Serialize};
+use std::{collections::BTreeMap, net::IpAddr};
+
+/// BTreeMap is used to store filters as the order of the filters will be stable, compared to
+/// storing them in a HashMap
+
+#[derive(Clone, Deserialize, Debug)]
+#[serde(rename_all = "kebab-case")]
+pub struct Answer {
+ pub global: Global,
+ pub network: Network,
+ #[serde(rename = "disk-setup")]
+ pub disks: Disks,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+pub struct Global {
+ pub country: String,
+ pub fqdn: Fqdn,
+ pub keyboard: String,
+ pub mailto: String,
+ pub timezone: String,
+ pub password: String,
+ pub pre_commands: Option<Vec<String>>,
+ pub post_commands: Option<Vec<String>>,
+ #[serde(default)]
+ pub reboot_on_error: bool,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+struct NetworkInAnswer {
+ #[serde(default)]
+ pub use_dhcp: bool,
+ pub cidr: Option<CidrAddress>,
+ pub dns: Option<IpAddr>,
+ pub gateway: Option<IpAddr>,
+ pub filter: Option<BTreeMap<String, String>>,
+}
+
+#[derive(Clone, Deserialize, Debug)]
+#[serde(try_from = "NetworkInAnswer")]
+pub struct Network {
+ pub network_settings: NetworkSettings,
+}
+
+impl TryFrom<NetworkInAnswer> for Network {
+ type Error = &'static str;
+
+ fn try_from(source: NetworkInAnswer) -> Result<Self, Self::Error> {
+ if !source.use_dhcp {
+ if source.cidr.is_none() {
+ return Err("Field 'cidr' must be set.");
+ }
+ if source.dns.is_none() {
+ return Err("Field 'dns' must be set.");
+ }
+ if source.gateway.is_none() {
+ return Err("Field 'gateway' must be set.");
+ }
+ if source.filter.is_none() {
+ return Err("Field 'filter' must be set.");
+ }
+
+ Ok(Network {
+ network_settings: NetworkSettings::Manual(NetworkManual {
+ cidr: source.cidr.unwrap(),
+ dns: source.dns.unwrap(),
+ gateway: source.gateway.unwrap(),
+ filter: source.filter.unwrap(),
+ }),
+ })
+ } else {
+ Ok(Network {
+ network_settings: NetworkSettings::Dhcp(true),
+ })
+ }
+ }
+}
+
+#[derive(Clone, Debug)]
+pub enum NetworkSettings {
+ Dhcp(bool),
+ Manual(NetworkManual),
+}
+
+#[derive(Clone, Debug)]
+pub struct NetworkManual {
+ pub cidr: CidrAddress,
+ pub dns: IpAddr,
+ pub gateway: IpAddr,
+ pub filter: BTreeMap<String, String>,
+}
+
+#[derive(Clone, Debug, Deserialize)]
+pub struct DiskSetup {
+ pub filesystem: Filesystem,
+ #[serde(default)]
+ pub disk_list: Vec<String>,
+ pub filter: Option<BTreeMap<String, String>>,
+ pub filter_match: Option<FilterMatch>,
+ pub zfs: Option<ZfsOptions>,
+ pub lvm: Option<LvmOptions>,
+ pub btrfs: Option<BtrfsOptions>,
+}
+
+#[derive(Clone, Debug, Deserialize)]
+#[serde(try_from = "DiskSetup")]
+pub struct Disks {
+ pub fs_type: FsType,
+ pub disk_selection: DiskSelection,
+ pub filter_match: Option<FilterMatch>,
+ pub fs_options: FsOptions,
+}
+
+impl TryFrom<DiskSetup> for Disks {
+ type Error = &'static str;
+
+ fn try_from(source: DiskSetup) -> Result<Self, Self::Error> {
+ if source.disk_list.is_empty() && source.filter.is_none() {
+ return Err("Need either 'disk_list' or 'filter' set");
+ }
+ if !source.disk_list.is_empty() && source.filter.is_some() {
+ return Err("Cannot use both, 'disk_list' and 'filter'");
+ }
+
+ let disk_selection = if !source.disk_list.is_empty() {
+ DiskSelection::Selection(source.disk_list.clone())
+ } else {
+ DiskSelection::Filter(source.filter.clone().unwrap())
+ };
+
+ let lvm_checks = |source: &DiskSetup| -> Result<(), Self::Error> {
+ if source.zfs.is_some() || source.btrfs.is_some() {
+ return Err("make sure only 'lvm' options are set");
+ }
+ if source.disk_list.len() > 1 {
+ return Err("make sure to define only one disk for ext4 and xfs");
+ }
+ Ok(())
+ };
+ // TODO: improve checks for foreign FS options. E.g. less verbose and handling new FS types
+ // automatically
+ let (fs, fs_options) = match source.filesystem {
+ Filesystem::Xfs => {
+ lvm_checks(&source)?;
+ (
+ FsType::Xfs,
+ FsOptions::LVM(source.lvm.unwrap_or(LvmOptions::default())),
+ )
+ }
+ Filesystem::Ext4 => {
+ lvm_checks(&source)?;
+ (
+ FsType::Ext4,
+ FsOptions::LVM(source.lvm.unwrap_or(LvmOptions::default())),
+ )
+ }
+ Filesystem::Zfs => {
+ if source.lvm.is_some() || source.btrfs.is_some() {
+ return Err("make sure only 'zfs' options are set");
+ }
+ match source.zfs {
+ None | Some(ZfsOptions { raid: None, .. }) => {
+ return Err("ZFS raid level 'zfs.raid' must be set")
+ }
+ Some(opts) => (FsType::Zfs(opts.raid.unwrap()), FsOptions::ZFS(opts)),
+ }
+ }
+ Filesystem::Btrfs => {
+ if source.zfs.is_some() || source.lvm.is_some() {
+ return Err("make sure only 'btrfs' options are set");
+ }
+ match source.btrfs {
+ None | Some(BtrfsOptions { raid: None, .. }) => {
+ return Err("BTRFS raid level 'btrfs.raid' must be set")
+ }
+ Some(opts) => (FsType::Btrfs(opts.raid.unwrap()), FsOptions::BTRFS(opts)),
+ }
+ }
+ };
+
+ let res = Disks {
+ fs_type: fs,
+ disk_selection,
+ filter_match: source.filter_match,
+ fs_options,
+ };
+ Ok(res)
+ }
+}
+
+#[derive(Clone, Debug)]
+pub enum FsOptions {
+ LVM(LvmOptions),
+ ZFS(ZfsOptions),
+ BTRFS(BtrfsOptions),
+}
+
+#[derive(Clone, Debug)]
+pub enum DiskSelection {
+ Selection(Vec<String>),
+ Filter(BTreeMap<String, String>),
+}
+#[derive(Clone, Deserialize, Debug, PartialEq)]
+#[serde(rename_all = "lowercase")]
+pub enum FilterMatch {
+ Any,
+ All,
+}
+
+#[derive(Clone, Deserialize, Serialize, Debug, PartialEq)]
+#[serde(rename_all = "lowercase")]
+pub enum Filesystem {
+ Ext4,
+ Xfs,
+ Zfs,
+ Btrfs,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Debug)]
+pub struct ZfsOptions {
+ pub raid: Option<ZfsRaidLevel>,
+ pub ashift: Option<usize>,
+ pub arc_max: Option<usize>,
+ pub checksum: Option<ZfsChecksumOption>,
+ pub compress: Option<ZfsCompressOption>,
+ pub copies: Option<usize>,
+ pub hdsize: Option<f64>,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Serialize, Debug)]
+pub struct LvmOptions {
+ pub hdsize: Option<f64>,
+ pub swapsize: Option<f64>,
+ pub maxroot: Option<f64>,
+ pub maxvz: Option<f64>,
+ pub minfree: Option<f64>,
+}
+
+#[derive(Clone, Copy, Default, Deserialize, Debug)]
+pub struct BtrfsOptions {
+ pub hdsize: Option<f64>,
+ pub raid: Option<BtrfsRaidLevel>,
+}
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index e69de29..7813b98 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -0,0 +1 @@
+pub mod answer;
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 10/30] auto-installer: add struct to hold udev info
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (8 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 09/30] auto-installer: add answer file definition Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 11/30] auto-installer: add utils Aaron Lauterer
` (21 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/udevinfo.rs | 9 +++++++++
2 files changed, 10 insertions(+)
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 7813b98..8cda416 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1 +1,2 @@
pub mod answer;
+pub mod udevinfo;
diff --git a/proxmox-auto-installer/src/udevinfo.rs b/proxmox-auto-installer/src/udevinfo.rs
new file mode 100644
index 0000000..a6b61b5
--- /dev/null
+++ b/proxmox-auto-installer/src/udevinfo.rs
@@ -0,0 +1,9 @@
+use serde::Deserialize;
+use std::collections::BTreeMap;
+
+#[derive(Clone, Deserialize, Debug)]
+pub struct UdevInfo {
+ // use BTreeMap to have keys sorted
+ pub disks: BTreeMap<String, BTreeMap<String, String>>,
+ pub nics: BTreeMap<String, BTreeMap<String, String>>,
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 11/30] auto-installer: add utils
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (9 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 10/30] auto-installer: add struct to hold udev info Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 12/30] auto-installer: add simple logging Aaron Lauterer
` (20 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
contains several utility structs and functions.
For example: a simple pattern matcher that matches wildcards at the
beginning or end of the filter.
It currently uses a dedicated function (parse_answer) to generate the
InstallConfig struct instead of a From implementation. This is because
for now the source data is spread over several other structs in
comparison to one in the TUI installer.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/utils.rs | 424 ++++++++++++++++++++++++++++
2 files changed, 425 insertions(+)
create mode 100644 proxmox-auto-installer/src/utils.rs
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 8cda416..72884c1 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,2 +1,3 @@
pub mod answer;
pub mod udevinfo;
+pub mod utils;
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
new file mode 100644
index 0000000..ae28b1e
--- /dev/null
+++ b/proxmox-auto-installer/src/utils.rs
@@ -0,0 +1,424 @@
+use anyhow::{bail, Result};
+use log::info;
+use std::{
+ collections::BTreeMap,
+ process::{Command, Stdio},
+};
+
+use crate::{
+ answer::{self, Answer},
+ udevinfo::UdevInfo,
+};
+use proxmox_installer_common::{
+ options::{FsType, NetworkOptions, ZfsChecksumOption, ZfsCompressOption},
+ setup::{InstallConfig, InstallZfsOption, LocaleInfo, RuntimeInfo, SetupInfo},
+};
+use serde::Deserialize;
+
+/// Supports the globbing character '*' at the beginning, end or both of the pattern.
+/// Globbing within the pattern is not supported
+fn find_with_glob(pattern: &str, value: &str) -> bool {
+ let globbing_symbol = '*';
+ let mut start_glob = false;
+ let mut end_glob = false;
+ let mut pattern = pattern;
+
+ if pattern.starts_with(globbing_symbol) {
+ start_glob = true;
+ pattern = &pattern[1..];
+ }
+
+ if pattern.ends_with(globbing_symbol) {
+ end_glob = true;
+ pattern = &pattern[..pattern.len() - 1]
+ }
+
+ match (start_glob, end_glob) {
+ (true, true) => value.contains(pattern),
+ (true, false) => value.ends_with(pattern),
+ (false, true) => value.starts_with(pattern),
+ _ => value == pattern,
+ }
+}
+
+pub fn get_network_settings(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ setup_info: &SetupInfo,
+) -> Result<NetworkOptions> {
+ let mut network_options = NetworkOptions::defaults_from(setup_info, &runtime_info.network);
+
+ info!("Setting network configuration");
+
+ // Always use the FQDN from the answer file
+ network_options.fqdn = answer.global.fqdn.clone();
+
+ if let answer::NetworkSettings::Manual(settings) = &answer.network.network_settings {
+ network_options.address = settings.cidr.clone();
+ network_options.dns_server = settings.dns;
+ network_options.gateway = settings.gateway;
+ network_options.ifname = get_single_udev_index(settings.filter.clone(), &udev_info.nics)?;
+ }
+ info!("Network interface used is '{}'", &network_options.ifname);
+ Ok(network_options)
+}
+
+fn get_single_udev_index(
+ filter: BTreeMap<String, String>,
+ udev_list: &BTreeMap<String, BTreeMap<String, String>>,
+) -> Result<String> {
+ if filter.is_empty() {
+ bail!("no filter defined");
+ }
+ let mut dev_index: Option<String> = None;
+ 'outer: for (dev, dev_values) in udev_list {
+ for (filter_key, filter_value) in &filter {
+ for (udev_key, udev_value) in dev_values {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ dev_index = Some(dev.clone());
+ break 'outer; // take first match
+ }
+ }
+ }
+ }
+ if dev_index.is_none() {
+ bail!("filter did not match any device");
+ }
+
+ Ok(dev_index.unwrap())
+}
+
+fn get_matched_udev_indexes(
+ filter: BTreeMap<String, String>,
+ udev_list: &BTreeMap<String, BTreeMap<String, String>>,
+ match_all: bool,
+) -> Result<Vec<String>> {
+ let mut matches = vec![];
+ for (dev, dev_values) in udev_list {
+ let mut did_match_once = false;
+ let mut did_match_all = true;
+ for (filter_key, filter_value) in &filter {
+ for (udev_key, udev_value) in dev_values {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ did_match_once = true;
+ } else if udev_key == filter_key {
+ did_match_all = false;
+ }
+ }
+ }
+ if (match_all && did_match_all) || (!match_all && did_match_once) {
+ matches.push(dev.clone());
+ }
+ }
+ if matches.is_empty() {
+ bail!("filter did not match any devices");
+ }
+ matches.sort();
+ Ok(matches)
+}
+
+pub fn set_disks(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match config.filesys {
+ FsType::Ext4 | FsType::Xfs => set_single_disk(answer, udev_info, runtime_info, config),
+ FsType::Zfs(_) | FsType::Btrfs(_) => {
+ set_selected_disks(answer, udev_info, runtime_info, config)
+ }
+ }
+}
+
+fn set_single_disk(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match &answer.disks.disk_selection {
+ answer::DiskSelection::Selection(disk_list) => {
+ let disk_name = disk_list[0].clone();
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.path.ends_with(disk_name.as_str()));
+ match disk {
+ Some(disk) => config.target_hd = Some(disk.clone()),
+ None => bail!("disk in 'disk_selection' not found"),
+ }
+ }
+ answer::DiskSelection::Filter(filter) => {
+ let disk_index = get_single_udev_index(filter.clone(), &udev_info.disks)?;
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index == disk_index);
+ config.target_hd = disk.cloned();
+ }
+ }
+ info!("Selected disk: {}", config.target_hd.clone().unwrap().path);
+ Ok(())
+}
+
+fn set_selected_disks(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ config: &mut InstallConfig,
+) -> Result<()> {
+ match &answer.disks.disk_selection {
+ answer::DiskSelection::Selection(disk_list) => {
+ info!("Disk selection found");
+ for disk_name in disk_list.clone() {
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.path.ends_with(disk_name.as_str()));
+ if let Some(disk) = disk {
+ config
+ .disk_selection
+ .insert(disk.index.clone(), disk.index.clone());
+ }
+ }
+ }
+ answer::DiskSelection::Filter(filter) => {
+ info!("No disk list found, looking for disk filters");
+ let filter_match = answer
+ .disks
+ .filter_match
+ .clone()
+ .unwrap_or(answer::FilterMatch::Any);
+ let disk_filters = filter.clone();
+ let selected_disk_indexes = get_matched_udev_indexes(
+ disk_filters,
+ &udev_info.disks,
+ filter_match == answer::FilterMatch::All,
+ )?;
+
+ for i in selected_disk_indexes.into_iter() {
+ let disk = runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index == i)
+ .unwrap();
+ config
+ .disk_selection
+ .insert(disk.index.clone(), disk.index.clone());
+ }
+ }
+ }
+ if config.disk_selection.is_empty() {
+ bail!("No disks found matching selection.");
+ }
+
+ let mut selected_disks: Vec<String> = Vec::new();
+ for i in config.disk_selection.keys() {
+ selected_disks.push(
+ runtime_info
+ .disks
+ .iter()
+ .find(|item| item.index.as_str() == i)
+ .unwrap()
+ .clone()
+ .path,
+ );
+ }
+ info!(
+ "Selected disks: {}",
+ selected_disks
+ .iter()
+ .map(|x| x.to_string() + " ")
+ .collect::<String>()
+ );
+
+ Ok(())
+}
+
+pub fn get_first_selected_disk(config: &InstallConfig) -> usize {
+ config
+ .disk_selection
+ .iter()
+ .next()
+ .expect("no disks found")
+ .0
+ .parse::<usize>()
+ .expect("could not parse key to usize")
+}
+
+pub fn verify_locale_settings(answer: &Answer, locales: &LocaleInfo) -> Result<()> {
+ info!("Verifying locale settings");
+ if !locales
+ .countries
+ .keys()
+ .any(|i| i == &answer.global.country)
+ {
+ bail!("country code '{}' is not valid", &answer.global.country);
+ }
+ if !locales.kmap.keys().any(|i| i == &answer.global.keyboard) {
+ bail!("keyboard layout '{}' is not valid", &answer.global.keyboard);
+ }
+ if !locales
+ .cczones
+ .iter()
+ .any(|(_, zones)| zones.contains(&answer.global.timezone))
+ {
+ bail!("timezone '{}' is not valid", &answer.global.timezone);
+ }
+ Ok(())
+}
+
+pub fn run_cmds(step: &str, cmd_vec: &Option<Vec<String>>) -> Result<()> {
+ if let Some(cmds) = cmd_vec {
+ if !cmds.is_empty() {
+ info!("Running {step}-Commands:");
+ run_cmd(cmds)?;
+ info!("{step}-Commands finished");
+ }
+ }
+ Ok(())
+}
+
+fn run_cmd(cmds: &Vec<String>) -> Result<()> {
+ for cmd in cmds {
+ info!("Command '{cmd}':");
+ let child = match Command::new("/bin/bash")
+ .arg("-c")
+ .arg(cmd.clone())
+ .stdout(Stdio::piped())
+ .stderr(Stdio::piped())
+ .spawn()
+ {
+ Ok(child) => child,
+ Err(err) => bail!("error running command {cmd}: {err}"),
+ };
+ match child.wait_with_output() {
+ Ok(output) => {
+ if output.status.success() {
+ info!("{}", String::from_utf8(output.stdout).unwrap());
+ } else {
+ bail!("{}", String::from_utf8(output.stderr).unwrap());
+ }
+ },
+ Err(err) => bail!("{err}"),
+ }
+ }
+
+ Ok(())
+}
+
+pub fn parse_answer(
+ answer: &Answer,
+ udev_info: &UdevInfo,
+ runtime_info: &RuntimeInfo,
+ locales: &LocaleInfo,
+ setup_info: &SetupInfo,
+) -> Result<InstallConfig> {
+ info!("Parsing answer file");
+ info!("Setting File system");
+ let filesystem = answer.disks.fs_type;
+ info!("File system selected: {}", filesystem);
+
+ let network_settings = get_network_settings(answer, udev_info, runtime_info, setup_info)?;
+
+ verify_locale_settings(answer, locales)?;
+
+ let mut config = InstallConfig {
+ autoreboot: 1_usize,
+ filesys: filesystem,
+ hdsize: 0.,
+ swapsize: None,
+ maxroot: None,
+ minfree: None,
+ maxvz: None,
+ zfs_opts: None,
+ target_hd: None,
+ disk_selection: BTreeMap::new(),
+
+ country: answer.global.country.clone(),
+ timezone: answer.global.timezone.clone(),
+ keymap: answer.global.keyboard.clone(),
+
+ password: answer.global.password.clone(),
+ mailto: answer.global.mailto.clone(),
+
+ mngmt_nic: network_settings.ifname,
+
+ hostname: network_settings.fqdn.host().unwrap().to_string(),
+ domain: network_settings.fqdn.domain(),
+ cidr: network_settings.address,
+ gateway: network_settings.gateway,
+ dns: network_settings.dns_server,
+ };
+
+ set_disks(answer, udev_info, runtime_info, &mut config)?;
+ match &answer.disks.fs_options {
+ answer::FsOptions::LVM(lvm) => {
+ config.hdsize = lvm.hdsize.unwrap_or(config.target_hd.clone().unwrap().size);
+ config.swapsize = lvm.swapsize;
+ config.maxroot = lvm.maxroot;
+ config.maxvz = lvm.maxvz;
+ config.minfree = lvm.minfree;
+ }
+ answer::FsOptions::ZFS(zfs) => {
+ let first_selected_disk = get_first_selected_disk(&config);
+
+ config.hdsize = zfs
+ .hdsize
+ .unwrap_or(runtime_info.disks[first_selected_disk].size);
+ config.zfs_opts = Some(InstallZfsOption {
+ ashift: zfs.ashift.unwrap_or(12),
+ arc_max: zfs.arc_max.unwrap_or(2048),
+ compress: zfs.compress.unwrap_or(ZfsCompressOption::On),
+ checksum: zfs.checksum.unwrap_or(ZfsChecksumOption::On),
+ copies: zfs.copies.unwrap_or(1),
+ });
+ }
+ answer::FsOptions::BTRFS(btrfs) => {
+ let first_selected_disk = get_first_selected_disk(&config);
+
+ config.hdsize = btrfs
+ .hdsize
+ .unwrap_or(runtime_info.disks[first_selected_disk].size);
+ }
+ }
+ Ok(config)
+}
+
+#[derive(Clone, Debug, Deserialize, PartialEq)]
+#[serde(tag = "type", rename_all = "lowercase")]
+pub enum LowLevelMessage {
+ #[serde(rename = "message")]
+ Info {
+ message: String,
+ },
+ Error {
+ message: String,
+ },
+ Prompt {
+ query: String,
+ },
+ Finished {
+ state: String,
+ message: String,
+ },
+ Progress {
+ ratio: f32,
+ text: String,
+ },
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ #[test]
+ fn test_glob_patterns() {
+ let test_value = "foobar";
+ assert_eq!(find_with_glob("*bar", test_value), true);
+ assert_eq!(find_with_glob("foo*", test_value), true);
+ assert_eq!(find_with_glob("foobar", test_value), true);
+ assert_eq!(find_with_glob("oobar", test_value), false);
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 12/30] auto-installer: add simple logging
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (10 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 11/30] auto-installer: add utils Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 13/30] auto-installer: add tests for answer file parsing Aaron Lauterer
` (19 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Log to stdout and the file the binary needs to set up.
This is a first variant. By using the log crate macros we can change
that in the future without too much effort.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 2 ++
proxmox-auto-installer/src/lib.rs | 1 +
proxmox-auto-installer/src/log.rs | 38 +++++++++++++++++++++++++++++++
3 files changed, 41 insertions(+)
create mode 100644 proxmox-auto-installer/src/log.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 80de4fa..b1d3e7a 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -8,8 +8,10 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+anyhow = "1.0"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.7"
enum-iterator = "0.6.0"
+log = "0.4.20"
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 72884c1..6636cc7 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,3 +1,4 @@
pub mod answer;
+pub mod log;
pub mod udevinfo;
pub mod utils;
diff --git a/proxmox-auto-installer/src/log.rs b/proxmox-auto-installer/src/log.rs
new file mode 100644
index 0000000..d52912a
--- /dev/null
+++ b/proxmox-auto-installer/src/log.rs
@@ -0,0 +1,38 @@
+use anyhow::{bail, Result};
+use log::{Level, Metadata, Record};
+use std::{fs::File, io::Write, sync::Mutex, sync::OnceLock};
+
+pub struct AutoInstLogger;
+static LOGFILE: OnceLock<Mutex<File>> = OnceLock::new();
+
+impl AutoInstLogger {
+ pub fn init(path: &str) -> Result<()> {
+ let f = File::create(path)?;
+ if LOGFILE.set(Mutex::new(f)).is_err() {
+ bail!("Cannot set LOGFILE")
+ }
+ Ok(())
+ }
+}
+
+impl log::Log for AutoInstLogger {
+ fn enabled(&self, metadata: &Metadata) -> bool {
+ metadata.level() <= Level::Info
+ }
+
+ /// Logs to stdout without log level and into log file including log level
+ fn log(&self, record: &Record) {
+ if self.enabled(record.metadata()) {
+ println!("{}", record.args());
+ let mut file = LOGFILE
+ .get()
+ .expect("could not get LOGFILE")
+ .lock()
+ .expect("could not get mutex for LOGFILE");
+ file.write_all(format!("{} - {}\n", record.level(), record.args()).as_bytes())
+ .expect("could not write to LOGFILE");
+ }
+ }
+
+ fn flush(&self) {}
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 13/30] auto-installer: add tests for answer file parsing
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (11 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 12/30] auto-installer: add simple logging Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 14/30] auto-installer: add auto-installer binary Aaron Lauterer
` (18 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=yes, Size: 90779 bytes --]
By matching the resulting json to be passed to the low level installer
against known good ones.
The environment info was gathered from one of our AMD Epyc Rome test
servers to have a realistic starting point.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/tests/parse-answer.rs | 106 ++++++++++++++++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 +++++
.../resources/parse_answer/disk_match.toml | 17 +++
.../parse_answer/disk_match_all.json | 26 +++++
.../parse_answer/disk_match_all.toml | 17 +++
.../parse_answer/disk_match_any.json | 33 ++++++
.../parse_answer/disk_match_any.toml | 17 +++
.../tests/resources/parse_answer/minimal.json | 17 +++
.../tests/resources/parse_answer/minimal.toml | 14 +++
.../resources/parse_answer/nic_matching.json | 17 +++
.../resources/parse_answer/nic_matching.toml | 19 ++++
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +++
.../resources/parse_answer/specific_nic.toml | 19 ++++
.../tests/resources/parse_answer/zfs.json | 27 +++++
.../tests/resources/parse_answer/zfs.toml | 20 ++++
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
20 files changed, 403 insertions(+)
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs
new file mode 100644
index 0000000..c12520f
--- /dev/null
+++ b/proxmox-auto-installer/tests/parse-answer.rs
@@ -0,0 +1,106 @@
+use std::path::PathBuf;
+
+use serde_json::Value;
+use std::fs;
+
+use proxmox_auto_installer::answer;
+use proxmox_auto_installer::answer::Answer;
+use proxmox_auto_installer::udevinfo::UdevInfo;
+use proxmox_auto_installer::utils::parse_answer;
+
+use proxmox_installer_common::setup::{read_json, LocaleInfo, RuntimeInfo, SetupInfo};
+
+fn get_test_resource_path() -> Result<PathBuf, String> {
+ Ok(std::env::current_dir()
+ .expect("current dir failed")
+ .join("tests/resources"))
+}
+fn get_answer(path: PathBuf) -> Result<Answer, String> {
+ let answer_raw = std::fs::read_to_string(&path).unwrap();
+ let answer: answer::Answer = toml::from_str(&answer_raw)
+ .map_err(|err| format!("error parsing answer.toml: {err}"))
+ .unwrap();
+
+ Ok(answer)
+}
+
+fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
+ let installer_info: SetupInfo = {
+ let mut path = path.clone();
+ path.push("iso-info.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve setup info: {err}"))
+ .unwrap()
+ };
+
+ let locale_info = {
+ let mut path = path.clone();
+ path.push("locales.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve locale info: {err}"))
+ .unwrap()
+ };
+
+ let mut runtime_info: RuntimeInfo = {
+ let mut path = path.clone();
+ path.push("run-env-info.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve runtime environment info: {err}"))
+ .unwrap()
+ };
+
+ let udev_info: UdevInfo = {
+ let mut path = path.clone();
+ path.push("run-env-udev.json");
+
+ read_json(&path)
+ .map_err(|err| format!("Failed to retrieve udev info details: {err}"))
+ .unwrap()
+ };
+ runtime_info.disks.sort();
+ if runtime_info.disks.is_empty() {
+ panic!("disk list is empty!");
+ }
+ (installer_info, locale_info, runtime_info, udev_info)
+}
+
+#[test]
+fn test_parse_answers() {
+ let path = get_test_resource_path().unwrap();
+ let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path);
+ let mut tests_path = path.clone();
+ tests_path.push("parse_answer");
+ let test_dir = fs::read_dir(tests_path.clone()).unwrap();
+
+ for file_entry in test_dir {
+ let file = file_entry.unwrap();
+ if !file.file_type().unwrap().is_file() || file.file_name() == "readme" {
+ continue;
+ }
+ let p = file.path();
+ let name = p.file_stem().unwrap().to_str().unwrap();
+ let extension = p.extension().unwrap().to_str().unwrap();
+ if extension == "toml" {
+ println!("Test: {name}");
+ let answer = get_answer(p.clone()).unwrap();
+ let config =
+ &parse_answer(&answer, &udev_info, &runtime_info, &locales, &setup_info).unwrap();
+ println!("Selected disks: {:#?}", &config.disk_selection);
+ let config_json = serde_json::to_string(config);
+ let config: Value = serde_json::from_str(config_json.unwrap().as_str()).unwrap();
+ let mut path = tests_path.clone();
+ path.push(format!("{name}.json"));
+ let compare_raw = std::fs::read_to_string(&path).unwrap();
+ let compare: Value = serde_json::from_str(&compare_raw).unwrap();
+ if config != compare {
+ panic!(
+ "Test {} failed:\nleft: {:#?}\nright: {:#?}\n",
+ name, config, compare
+ );
+ }
+ }
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/iso-info.json b/proxmox-auto-installer/tests/resources/iso-info.json
new file mode 100644
index 0000000..33cb79b
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/iso-info.json
@@ -0,0 +1 @@
+{"iso-info":{"isoname":"proxmox-ve","isorelease":"2","product":"pve","productlong":"Proxmox VE","release":"8.0"},"locations":{"iso":"/cdrom","lib":"/var/lib/proxmox-installer","pkg":"/cdrom/proxmox/packages/","run":"/run/proxmox-installer"},"product":"pve","product-cfg":{"bridged_network":1,"enable_btrfs":1,"fullname":"Proxmox VE","port":"8006","product":"pve"},"run-env-cache-file":"/run/proxmox-installer/run-env-info.json"}
diff --git a/proxmox-auto-installer/tests/resources/locales.json b/proxmox-auto-installer/tests/resources/locales.json
new file mode 100644
index 0000000..220a18c
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/locales.json
@@ -0,0 +1 @@
+{"cczones":{"ad":{"Europe/Andorra":1},"ae":{"Asia/Dubai":1},"af":{"Asia/Kabul":1},"ag":{"America/Antigua":1},"ai":{"America/Anguilla":1},"al":{"Europe/Tirane":1},"am":{"Asia/Yerevan":1},"ao":{"Africa/Luanda":1},"aq":{"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1},"ar":{"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1},"as":{"Pacific/Pago_Pago":1},"at":{"Europe/Vienna":1},"au":{"Antarctica/Macquarie":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1},"aw":{"America/Aruba":1},"ax":{"Europe/Mariehamn":1},"az":{"Asia/Baku":1},"ba":{"Europe/Sarajevo":1},"bb":{"America/Barbados":1},"bd":{"Asia/Dhaka":1},"be":{"Europe/Brussels":1},"bf":{"Africa/Ouagadougou":1},"bg":{"Europe/Sofia":1},"bh":{"Asia/Bahrain":1},"bi":{"Africa/Bujumbura":1},"bj":{"Africa/Porto-Novo":1},"bl":{"America/St_Barthelemy":1},"bm":{"Atlantic/Bermuda":1},"bn":{"Asia/Brunei":1},"bo":{"America/La_Paz":1},"bq":{"America/Kralendijk":1},"br":{"America/Araguaina":1,"America/Bahia":1,"America/Belem":1,"America/Boa_Vista":1,"America/Campo_Grande":1,"America/Cuiaba":1,"America/Eirunepe":1,"America/Fortaleza":1,"America/Maceio":1,"America/Manaus":1,"America/Noronha":1,"America/Porto_Velho":1,"America/Recife":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Sao_Paulo":1},"bs":{"America/Nassau":1},"bt":{"Asia/Thimphu":1},"bw":{"Africa/Gaborone":1},"by":{"Europe/Minsk":1},"bz":{"America/Belize":1},"ca":{"America/Atikokan":1,"America/Blanc-Sablon":1,"America/Cambridge_Bay":1,"America/Creston":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Edmonton":1,"America/Fort_Nelson":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Halifax":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Moncton":1,"America/Rankin_Inlet":1,"America/Regina":1,"America/Resolute":1,"America/St_Johns":1,"America/Swift_Current":1,"America/Toronto":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1},"cc":{"Indian/Cocos":1},"cd":{"Africa/Kinshasa":1,"Africa/Lubumbashi":1},"cf":{"Africa/Bangui":1},"cg":{"Africa/Brazzaville":1},"ch":{"Europe/Zurich":1},"ci":{"Africa/Abidjan":1},"ck":{"Pacific/Rarotonga":1},"cl":{"America/Punta_Arenas":1,"America/Santiago":1,"Pacific/Easter":1},"cm":{"Africa/Douala":1},"cn":{"Asia/Shanghai":1,"Asia/Urumqi":1},"co":{"America/Bogota":1},"cr":{"America/Costa_Rica":1},"cu":{"America/Havana":1},"cv":{"Atlantic/Cape_Verde":1},"cw":{"America/Curacao":1},"cx":{"Indian/Christmas":1},"cy":{"Asia/Famagusta":1,"Asia/Nicosia":1},"cz":{"Europe/Prague":1},"de":{"Europe/Berlin":1,"Europe/Busingen":1},"dj":{"Africa/Djibouti":1},"dk":{"Europe/Copenhagen":1},"dm":{"America/Dominica":1},"do":{"America/Santo_Domingo":1},"dz":{"Africa/Algiers":1},"ec":{"America/Guayaquil":1,"Pacific/Galapagos":1},"ee":{"Europe/Tallinn":1},"eg":{"Africa/Cairo":1},"eh":{"Africa/El_Aaiun":1},"er":{"Africa/Asmara":1},"es":{"Africa/Ceuta":1,"Atlantic/Canary":1,"Europe/Madrid":1},"et":{"Africa/Addis_Ababa":1},"fi":{"Europe/Helsinki":1},"fj":{"Pacific/Fiji":1},"fk":{"Atlantic/Stanley":1},"fm":{"Pacific/Chuuk":1,"Pacific/Kosrae":1,"Pacific/Pohnpei":1},"fo":{"Atlantic/Faroe":1},"fr":{"Europe/Paris":1},"ga":{"Africa/Libreville":1},"gb":{"Europe/London":1},"gd":{"America/Grenada":1},"ge":{"Asia/Tbilisi":1},"gf":{"America/Cayenne":1},"gg":{"Europe/Guernsey":1},"gh":{"Africa/Accra":1},"gi":{"Europe/Gibraltar":1},"gl":{"America/Danmarkshavn":1,"America/Nuuk":1,"America/Scoresbysund":1,"America/Thule":1},"gm":{"Africa/Banjul":1},"gn":{"Africa/Conakry":1},"gp":{"America/Guadeloupe":1},"gq":{"Africa/Malabo":1},"gr":{"Europe/Athens":1},"gs":{"Atlantic/South_Georgia":1},"gt":{"America/Guatemala":1},"gu":{"Pacific/Guam":1},"gw":{"Africa/Bissau":1},"gy":{"America/Guyana":1},"hk":{"Asia/Hong_Kong":1},"hn":{"America/Tegucigalpa":1},"hr":{"Europe/Zagreb":1},"ht":{"America/Port-au-Prince":1},"hu":{"Europe/Budapest":1},"id":{"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Makassar":1,"Asia/Pontianak":1},"ie":{"Europe/Dublin":1},"il":{"Asia/Jerusalem":1},"im":{"Europe/Isle_of_Man":1},"in":{"Asia/Kolkata":1},"io":{"Indian/Chagos":1},"iq":{"Asia/Baghdad":1},"ir":{"Asia/Tehran":1},"is":{"Atlantic/Reykjavik":1},"it":{"Europe/Rome":1},"je":{"Europe/Jersey":1},"jm":{"America/Jamaica":1},"jo":{"Asia/Amman":1},"jp":{"Asia/Tokyo":1},"ke":{"Africa/Nairobi":1},"kg":{"Asia/Bishkek":1},"kh":{"Asia/Phnom_Penh":1},"ki":{"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Tarawa":1},"km":{"Indian/Comoro":1},"kn":{"America/St_Kitts":1},"kp":{"Asia/Pyongyang":1},"kr":{"Asia/Seoul":1},"kw":{"Asia/Kuwait":1},"ky":{"America/Cayman":1},"kz":{"Asia/Almaty":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Atyrau":1,"Asia/Oral":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1},"la":{"Asia/Vientiane":1},"lb":{"Asia/Beirut":1},"lc":{"America/St_Lucia":1},"li":{"Europe/Vaduz":1},"lk":{"Asia/Colombo":1},"lr":{"Africa/Monrovia":1},"ls":{"Africa/Maseru":1},"lt":{"Europe/Vilnius":1},"lu":{"Europe/Luxembourg":1},"lv":{"Europe/Riga":1},"ly":{"Africa/Tripoli":1},"ma":{"Africa/Casablanca":1},"mc":{"Europe/Monaco":1},"md":{"Europe/Chisinau":1},"me":{"Europe/Podgorica":1},"mf":{"America/Marigot":1},"mg":{"Indian/Antananarivo":1},"mh":{"Pacific/Kwajalein":1,"Pacific/Majuro":1},"mk":{"Europe/Skopje":1},"ml":{"Africa/Bamako":1},"mm":{"Asia/Yangon":1},"mn":{"Asia/Choibalsan":1,"Asia/Hovd":1,"Asia/Ulaanbaatar":1},"mo":{"Asia/Macau":1},"mp":{"Pacific/Saipan":1},"mq":{"America/Martinique":1},"mr":{"Africa/Nouakchott":1},"ms":{"America/Montserrat":1},"mt":{"Europe/Malta":1},"mu":{"Indian/Mauritius":1},"mv":{"Indian/Maldives":1},"mw":{"Africa/Blantyre":1},"mx":{"America/Bahia_Banderas":1,"America/Cancun":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Hermosillo":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Merida":1,"America/Mexico_City":1,"America/Monterrey":1,"America/Ojinaga":1,"America/Tijuana":1},"my":{"Asia/Kuala_Lumpur":1,"Asia/Kuching":1},"mz":{"Africa/Maputo":1},"na":{"Africa/Windhoek":1},"nc":{"Pacific/Noumea":1},"ne":{"Africa/Niamey":1},"nf":{"Pacific/Norfolk":1},"ng":{"Africa/Lagos":1},"ni":{"America/Managua":1},"nl":{"Europe/Amsterdam":1},"no":{"Europe/Oslo":1},"np":{"Asia/Kathmandu":1},"nr":{"Pacific/Nauru":1},"nu":{"Pacific/Niue":1},"nz":{"Pacific/Auckland":1,"Pacific/Chatham":1},"om":{"Asia/Muscat":1},"pa":{"America/Panama":1},"pe":{"America/Lima":1},"pf":{"Pacific/Gambier":1,"Pacific/Marquesas":1,"Pacific/Tahiti":1},"pg":{"Pacific/Bougainville":1,"Pacific/Port_Moresby":1},"ph":{"Asia/Manila":1},"pk":{"Asia/Karachi":1},"pl":{"Europe/Warsaw":1},"pm":{"America/Miquelon":1},"pn":{"Pacific/Pitcairn":1},"pr":{"America/Puerto_Rico":1},"ps":{"Asia/Gaza":1,"Asia/Hebron":1},"pt":{"Atlantic/Azores":1,"Atlantic/Madeira":1,"Europe/Lisbon":1},"pw":{"Pacific/Palau":1},"py":{"America/Asuncion":1},"qa":{"Asia/Qatar":1},"re":{"Indian/Reunion":1},"ro":{"Europe/Bucharest":1},"rs":{"Europe/Belgrade":1},"ru":{"Asia/Anadyr":1,"Asia/Barnaul":1,"Asia/Chita":1,"Asia/Irkutsk":1,"Asia/Kamchatka":1,"Asia/Khandyga":1,"Asia/Krasnoyarsk":1,"Asia/Magadan":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Sakhalin":1,"Asia/Srednekolymsk":1,"Asia/Tomsk":1,"Asia/Ust-Nera":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yekaterinburg":1,"Europe/Astrakhan":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Moscow":1,"Europe/Samara":1,"Europe/Saratov":1,"Europe/Ulyanovsk":1,"Europe/Volgograd":1},"rw":{"Africa/Kigali":1},"sa":{"Asia/Riyadh":1},"sb":{"Pacific/Guadalcanal":1},"sc":{"Indian/Mahe":1},"sd":{"Africa/Khartoum":1},"se":{"Europe/Stockholm":1},"sg":{"Asia/Singapore":1},"sh":{"Atlantic/St_Helena":1},"si":{"Europe/Ljubljana":1},"sj":{"Arctic/Longyearbyen":1},"sk":{"Europe/Bratislava":1},"sl":{"Africa/Freetown":1},"sm":{"Europe/San_Marino":1},"sn":{"Africa/Dakar":1},"so":{"Africa/Mogadishu":1},"sr":{"America/Paramaribo":1},"ss":{"Africa/Juba":1},"st":{"Africa/Sao_Tome":1},"sv":{"America/El_Salvador":1},"sx":{"America/Lower_Princes":1},"sy":{"Asia/Damascus":1},"sz":{"Africa/Mbabane":1},"tc":{"America/Grand_Turk":1},"td":{"Africa/Ndjamena":1},"tf":{"Indian/Kerguelen":1},"tg":{"Africa/Lome":1},"th":{"Asia/Bangkok":1},"tj":{"Asia/Dushanbe":1},"tk":{"Pacific/Fakaofo":1},"tl":{"Asia/Dili":1},"tm":{"Asia/Ashgabat":1},"tn":{"Africa/Tunis":1},"to":{"Pacific/Tongatapu":1},"tr":{"Europe/Istanbul":1},"tt":{"America/Port_of_Spain":1},"tv":{"Pacific/Funafuti":1},"tw":{"Asia/Taipei":1},"tz":{"Africa/Dar_es_Salaam":1},"ua":{"Europe/Kyiv":1,"Europe/Simferopol":1},"ug":{"Africa/Kampala":1},"um":{"Pacific/Midway":1,"Pacific/Wake":1},"us":{"America/Adak":1,"America/Anchorage":1,"America/Boise":1,"America/Chicago":1,"America/Denver":1,"America/Detroit":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Los_Angeles":1,"America/Menominee":1,"America/Metlakatla":1,"America/New_York":1,"America/Nome":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Phoenix":1,"America/Sitka":1,"America/Yakutat":1,"Pacific/Honolulu":1},"uy":{"America/Montevideo":1},"uz":{"Asia/Samarkand":1,"Asia/Tashkent":1},"va":{"Europe/Vatican":1},"vc":{"America/St_Vincent":1},"ve":{"America/Caracas":1},"vg":{"America/Tortola":1},"vi":{"America/St_Thomas":1},"vn":{"Asia/Ho_Chi_Minh":1},"vu":{"Pacific/Efate":1},"wf":{"Pacific/Wallis":1},"ws":{"Pacific/Apia":1},"ye":{"Asia/Aden":1},"yt":{"Indian/Mayotte":1},"za":{"Africa/Johannesburg":1},"zm":{"Africa/Lusaka":1},"zw":{"Africa/Harare":1}},"country":{"ad":{"kmap":"","mirror":"","name":"Andorra","zone":"Europe/Andorra"},"ae":{"kmap":"","mirror":"","name":"United Arab Emirates","zone":"Asia/Dubai"},"af":{"kmap":"","mirror":"","name":"Afghanistan","zone":"Asia/Kabul"},"ag":{"kmap":"","mirror":"","name":"Antigua and Barbuda","zone":"America/Antigua"},"ai":{"kmap":"","mirror":"","name":"Anguilla","zone":"America/Anguilla"},"al":{"kmap":"","mirror":"","name":"Albania","zone":"Europe/Tirane"},"am":{"kmap":"","mirror":"","name":"Armenia","zone":"Asia/Yerevan"},"ao":{"kmap":"","mirror":"","name":"Angola","zone":"Africa/Luanda"},"aq":{"kmap":"","mirror":"","name":"Antarctica","zone":"Antarctica/McMurdo"},"ar":{"kmap":"","mirror":"","name":"Argentina","zone":"America/Argentina/Buenos_Aires"},"as":{"kmap":"","mirror":"","name":"American Samoa","zone":"Pacific/Pago_Pago"},"at":{"kmap":"de","mirror":"ftp.at.debian.org","name":"Austria","zone":"Europe/Vienna"},"au":{"kmap":"","mirror":"ftp.au.debian.org","name":"Australia","zone":"Australia/Lord_Howe"},"aw":{"kmap":"","mirror":"","name":"Aruba","zone":"America/Aruba"},"ax":{"kmap":"","mirror":"","name":"Åland Islands","zone":"Europe/Mariehamn"},"az":{"kmap":"","mirror":"","name":"Azerbaijan","zone":"Asia/Baku"},"ba":{"kmap":"","mirror":"","name":"Bosnia and Herzegovina","zone":"Europe/Sarajevo"},"bb":{"kmap":"","mirror":"","name":"Barbados","zone":"America/Barbados"},"bd":{"kmap":"","mirror":"","name":"Bangladesh","zone":"Asia/Dhaka"},"be":{"kmap":"fr-be","mirror":"ftp.be.debian.org","name":"Belgium","zone":"Europe/Brussels"},"bf":{"kmap":"","mirror":"","name":"Burkina Faso","zone":"Africa/Ouagadougou"},"bg":{"kmap":"","mirror":"ftp.bg.debian.org","name":"Bulgaria","zone":"Europe/Sofia"},"bh":{"kmap":"","mirror":"","name":"Bahrain","zone":"Asia/Bahrain"},"bi":{"kmap":"","mirror":"","name":"Burundi","zone":"Africa/Bujumbura"},"bj":{"kmap":"","mirror":"","name":"Benin","zone":"Africa/Porto-Novo"},"bl":{"kmap":"","mirror":"","name":"Saint Barthélemy","zone":"America/St_Barthelemy"},"bm":{"kmap":"","mirror":"","name":"Bermuda","zone":"Atlantic/Bermuda"},"bn":{"kmap":"","mirror":"","name":"Brunei Darussalam","zone":"Asia/Brunei"},"bo":{"kmap":"","mirror":"","name":"Bolivia","zone":"America/La_Paz"},"bq":{"kmap":"","mirror":"","name":"Bonaire, Sint Eustatius and Saba","zone":"America/Kralendijk"},"br":{"kmap":"pt-br","mirror":"ftp.br.debian.org","name":"Brazil","zone":"America/Noronha"},"bs":{"kmap":"","mirror":"","name":"Bahamas","zone":"America/Nassau"},"bt":{"kmap":"","mirror":"","name":"Bhutan","zone":"Asia/Thimphu"},"bv":{"kmap":"","mirror":"","name":"Bouvet Island"},"bw":{"kmap":"","mirror":"","name":"Botswana","zone":"Africa/Gaborone"},"by":{"kmap":"","mirror":"","name":"Belarus","zone":"Europe/Minsk"},"bz":{"kmap":"","mirror":"","name":"Belize","zone":"America/Belize"},"ca":{"kmap":"en-us","mirror":"ftp.ca.debian.org","name":"Canada","zone":"America/St_Johns"},"cc":{"kmap":"","mirror":"","name":"Cocos (Keeling) Islands","zone":"Indian/Cocos"},"cd":{"kmap":"","mirror":"","name":"Congo, The Democratic Republic of the","zone":"Africa/Kinshasa"},"cf":{"kmap":"","mirror":"","name":"Central African Republic","zone":"Africa/Bangui"},"cg":{"kmap":"","mirror":"","name":"Congo","zone":"Africa/Brazzaville"},"ch":{"kmap":"de-ch","mirror":"ftp.ch.debian.org","name":"Switzerland","zone":"Europe/Zurich"},"ci":{"kmap":"","mirror":"","name":"Côte d'Ivoire","zone":"Africa/Abidjan"},"ck":{"kmap":"","mirror":"","name":"Cook Islands","zone":"Pacific/Rarotonga"},"cl":{"kmap":"","mirror":"ftp.cl.debian.org","name":"Chile","zone":"America/Santiago"},"cm":{"kmap":"","mirror":"","name":"Cameroon","zone":"Africa/Douala"},"cn":{"kmap":"","mirror":"","name":"China","zone":"Asia/Shanghai"},"co":{"kmap":"","mirror":"","name":"Colombia","zone":"America/Bogota"},"cr":{"kmap":"","mirror":"","name":"Costa Rica","zone":"America/Costa_Rica"},"cu":{"kmap":"","mirror":"","name":"Cuba","zone":"America/Havana"},"cv":{"kmap":"","mirror":"","name":"Cabo Verde","zone":"Atlantic/Cape_Verde"},"cw":{"kmap":"","mirror":"","name":"Curaçao","zone":"America/Curacao"},"cx":{"kmap":"","mirror":"","name":"Christmas Island","zone":"Indian/Christmas"},"cy":{"kmap":"","mirror":"","name":"Cyprus","zone":"Asia/Nicosia"},"cz":{"kmap":"","mirror":"ftp.cz.debian.org","name":"Czechia","zone":"Europe/Prague"},"de":{"kmap":"de","mirror":"ftp.de.debian.org","name":"Germany","zone":"Europe/Berlin"},"dj":{"kmap":"","mirror":"","name":"Djibouti","zone":"Africa/Djibouti"},"dk":{"kmap":"dk","mirror":"ftp.dk.debian.org","name":"Denmark","zone":"Europe/Copenhagen"},"dm":{"kmap":"","mirror":"","name":"Dominica","zone":"America/Dominica"},"do":{"kmap":"","mirror":"","name":"Dominican Republic","zone":"America/Santo_Domingo"},"dz":{"kmap":"","mirror":"","name":"Algeria","zone":"Africa/Algiers"},"ec":{"kmap":"","mirror":"","name":"Ecuador","zone":"America/Guayaquil"},"ee":{"kmap":"","mirror":"ftp.ee.debian.org","name":"Estonia","zone":"Europe/Tallinn"},"eg":{"kmap":"","mirror":"","name":"Egypt","zone":"Africa/Cairo"},"eh":{"kmap":"","mirror":"","name":"Western Sahara","zone":"Africa/El_Aaiun"},"er":{"kmap":"","mirror":"","name":"Eritrea","zone":"Africa/Asmara"},"es":{"kmap":"es","mirror":"ftp.es.debian.org","name":"Spain","zone":"Europe/Madrid"},"et":{"kmap":"","mirror":"","name":"Ethiopia","zone":"Africa/Addis_Ababa"},"fi":{"kmap":"fi","mirror":"ftp.fi.debian.org","name":"Finland","zone":"Europe/Helsinki"},"fj":{"kmap":"","mirror":"","name":"Fiji","zone":"Pacific/Fiji"},"fk":{"kmap":"","mirror":"","name":"Falkland Islands (Malvinas)","zone":"Atlantic/Stanley"},"fm":{"kmap":"","mirror":"","name":"Micronesia, Federated States of","zone":"Pacific/Chuuk"},"fo":{"kmap":"","mirror":"","name":"Faroe Islands","zone":"Atlantic/Faroe"},"fr":{"kmap":"fr","mirror":"ftp.fr.debian.org","name":"France","zone":"Europe/Paris"},"ga":{"kmap":"","mirror":"","name":"Gabon","zone":"Africa/Libreville"},"gb":{"kmap":"en-gb","mirror":"ftp.uk.debian.org","name":"United Kingdom","zone":"Europe/London"},"gd":{"kmap":"","mirror":"","name":"Grenada","zone":"America/Grenada"},"ge":{"kmap":"","mirror":"","name":"Georgia","zone":"Asia/Tbilisi"},"gf":{"kmap":"","mirror":"","name":"French Guiana","zone":"America/Cayenne"},"gg":{"kmap":"","mirror":"","name":"Guernsey","zone":"Europe/Guernsey"},"gh":{"kmap":"","mirror":"","name":"Ghana","zone":"Africa/Accra"},"gi":{"kmap":"es","mirror":"","name":"Gibraltar","zone":"Europe/Gibraltar"},"gl":{"kmap":"","mirror":"","name":"Greenland","zone":"America/Nuuk"},"gm":{"kmap":"","mirror":"","name":"Gambia","zone":"Africa/Banjul"},"gn":{"kmap":"","mirror":"","name":"Guinea","zone":"Africa/Conakry"},"gp":{"kmap":"","mirror":"","name":"Guadeloupe","zone":"America/Guadeloupe"},"gq":{"kmap":"","mirror":"","name":"Equatorial Guinea","zone":"Africa/Malabo"},"gr":{"kmap":"","mirror":"ftp.gr.debian.org","name":"Greece","zone":"Europe/Athens"},"gs":{"kmap":"","mirror":"","name":"South Georgia and the South Sandwich Islands","zone":"Atlantic/South_Georgia"},"gt":{"kmap":"","mirror":"","name":"Guatemala","zone":"America/Guatemala"},"gu":{"kmap":"","mirror":"","name":"Guam","zone":"Pacific/Guam"},"gw":{"kmap":"","mirror":"","name":"Guinea-Bissau","zone":"Africa/Bissau"},"gy":{"kmap":"","mirror":"","name":"Guyana","zone":"America/Guyana"},"hk":{"kmap":"","mirror":"ftp.hk.debian.org","name":"Hong Kong","zone":"Asia/Hong_Kong"},"hm":{"kmap":"","mirror":"","name":"Heard Island and McDonald Islands"},"hn":{"kmap":"","mirror":"","name":"Honduras","zone":"America/Tegucigalpa"},"hr":{"kmap":"","mirror":"ftp.hr.debian.org","name":"Croatia","zone":"Europe/Zagreb"},"ht":{"kmap":"","mirror":"","name":"Haiti","zone":"America/Port-au-Prince"},"hu":{"kmap":"hu","mirror":"ftp.hu.debian.org","name":"Hungary","zone":"Europe/Budapest"},"id":{"kmap":"","mirror":"","name":"Indonesia","zone":"Asia/Jakarta"},"ie":{"kmap":"","mirror":"ftp.ie.debian.org","name":"Ireland","zone":"Europe/Dublin"},"il":{"kmap":"","mirror":"","name":"Israel","zone":"Asia/Jerusalem"},"im":{"kmap":"","mirror":"","name":"Isle of Man","zone":"Europe/Isle_of_Man"},"in":{"kmap":"","mirror":"","name":"India","zone":"Asia/Kolkata"},"io":{"kmap":"","mirror":"","name":"British Indian Ocean Territory","zone":"Indian/Chagos"},"iq":{"kmap":"","mirror":"","name":"Iraq","zone":"Asia/Baghdad"},"ir":{"kmap":"","mirror":"","name":"Iran","zone":"Asia/Tehran"},"is":{"kmap":"is","mirror":"ftp.is.debian.org","name":"Iceland","zone":"Atlantic/Reykjavik"},"it":{"kmap":"it","mirror":"ftp.it.debian.org","name":"Italy","zone":"Europe/Rome"},"je":{"kmap":"","mirror":"","name":"Jersey","zone":"Europe/Jersey"},"jm":{"kmap":"","mirror":"","name":"Jamaica","zone":"America/Jamaica"},"jo":{"kmap":"","mirror":"","name":"Jordan","zone":"Asia/Amman"},"jp":{"kmap":"jp","mirror":"ftp.jp.debian.org","name":"Japan","zone":"Asia/Tokyo"},"ke":{"kmap":"","mirror":"","name":"Kenya","zone":"Africa/Nairobi"},"kg":{"kmap":"","mirror":"","name":"Kyrgyzstan","zone":"Asia/Bishkek"},"kh":{"kmap":"","mirror":"","name":"Cambodia","zone":"Asia/Phnom_Penh"},"ki":{"kmap":"","mirror":"","name":"Kiribati","zone":"Pacific/Tarawa"},"km":{"kmap":"","mirror":"","name":"Comoros","zone":"Indian/Comoro"},"kn":{"kmap":"","mirror":"","name":"Saint Kitts and Nevis","zone":"America/St_Kitts"},"kp":{"kmap":"","mirror":"","name":"North Korea","zone":"Asia/Pyongyang"},"kr":{"kmap":"","mirror":"ftp.kr.debian.org","name":"South Korea","zone":"Asia/Seoul"},"kw":{"kmap":"","mirror":"","name":"Kuwait","zone":"Asia/Kuwait"},"ky":{"kmap":"","mirror":"","name":"Cayman Islands","zone":"America/Cayman"},"kz":{"kmap":"","mirror":"","name":"Kazakhstan","zone":"Asia/Almaty"},"la":{"kmap":"","mirror":"","name":"Laos","zone":"Asia/Vientiane"},"lb":{"kmap":"","mirror":"","name":"Lebanon","zone":"Asia/Beirut"},"lc":{"kmap":"","mirror":"","name":"Saint Lucia","zone":"America/St_Lucia"},"li":{"kmap":"de-ch","mirror":"","name":"Liechtenstein","zone":"Europe/Vaduz"},"lk":{"kmap":"","mirror":"","name":"Sri Lanka","zone":"Asia/Colombo"},"lr":{"kmap":"","mirror":"","name":"Liberia","zone":"Africa/Monrovia"},"ls":{"kmap":"","mirror":"","name":"Lesotho","zone":"Africa/Maseru"},"lt":{"kmap":"lt","mirror":"","name":"Lithuania","zone":"Europe/Vilnius"},"lu":{"kmap":"fr-ch","mirror":"","name":"Luxembourg","zone":"Europe/Luxembourg"},"lv":{"kmap":"","mirror":"","name":"Latvia","zone":"Europe/Riga"},"ly":{"kmap":"","mirror":"","name":"Libya","zone":"Africa/Tripoli"},"ma":{"kmap":"","mirror":"","name":"Morocco","zone":"Africa/Casablanca"},"mc":{"kmap":"","mirror":"","name":"Monaco","zone":"Europe/Monaco"},"md":{"kmap":"","mirror":"","name":"Moldova","zone":"Europe/Chisinau"},"me":{"kmap":"","mirror":"","name":"Montenegro","zone":"Europe/Podgorica"},"mf":{"kmap":"","mirror":"","name":"Saint Martin (French part)","zone":"America/Marigot"},"mg":{"kmap":"","mirror":"","name":"Madagascar","zone":"Indian/Antananarivo"},"mh":{"kmap":"","mirror":"","name":"Marshall Islands","zone":"Pacific/Majuro"},"mk":{"kmap":"mk","mirror":"","name":"North Macedonia","zone":"Europe/Skopje"},"ml":{"kmap":"","mirror":"","name":"Mali","zone":"Africa/Bamako"},"mm":{"kmap":"","mirror":"","name":"Myanmar","zone":"Asia/Yangon"},"mn":{"kmap":"","mirror":"","name":"Mongolia","zone":"Asia/Ulaanbaatar"},"mo":{"kmap":"","mirror":"","name":"Macao","zone":"Asia/Macau"},"mp":{"kmap":"","mirror":"","name":"Northern Mariana Islands","zone":"Pacific/Saipan"},"mq":{"kmap":"","mirror":"","name":"Martinique","zone":"America/Martinique"},"mr":{"kmap":"","mirror":"","name":"Mauritania","zone":"Africa/Nouakchott"},"ms":{"kmap":"","mirror":"","name":"Montserrat","zone":"America/Montserrat"},"mt":{"kmap":"","mirror":"","name":"Malta","zone":"Europe/Malta"},"mu":{"kmap":"","mirror":"","name":"Mauritius","zone":"Indian/Mauritius"},"mv":{"kmap":"","mirror":"","name":"Maldives","zone":"Indian/Maldives"},"mw":{"kmap":"","mirror":"","name":"Malawi","zone":"Africa/Blantyre"},"mx":{"kmap":"","mirror":"ftp.mx.debian.org","name":"Mexico","zone":"America/Mexico_City"},"my":{"kmap":"","mirror":"","name":"Malaysia","zone":"Asia/Kuala_Lumpur"},"mz":{"kmap":"","mirror":"","name":"Mozambique","zone":"Africa/Maputo"},"na":{"kmap":"","mirror":"","name":"Namibia","zone":"Africa/Windhoek"},"nc":{"kmap":"","mirror":"","name":"New Caledonia","zone":"Pacific/Noumea"},"ne":{"kmap":"","mirror":"","name":"Niger","zone":"Africa/Niamey"},"nf":{"kmap":"","mirror":"","name":"Norfolk Island","zone":"Pacific/Norfolk"},"ng":{"kmap":"","mirror":"","name":"Nigeria","zone":"Africa/Lagos"},"ni":{"kmap":"","mirror":"","name":"Nicaragua","zone":"America/Managua"},"nl":{"kmap":"en-us","mirror":"ftp.nl.debian.org","name":"Netherlands","zone":"Europe/Amsterdam"},"no":{"kmap":"no","mirror":"ftp.no.debian.org","name":"Norway","zone":"Europe/Oslo"},"np":{"kmap":"","mirror":"","name":"Nepal","zone":"Asia/Kathmandu"},"nr":{"kmap":"","mirror":"","name":"Nauru","zone":"Pacific/Nauru"},"nu":{"kmap":"","mirror":"","name":"Niue","zone":"Pacific/Niue"},"nz":{"kmap":"","mirror":"ftp.nz.debian.org","name":"New Zealand","zone":"Pacific/Auckland"},"om":{"kmap":"","mirror":"","name":"Oman","zone":"Asia/Muscat"},"pa":{"kmap":"","mirror":"","name":"Panama","zone":"America/Panama"},"pe":{"kmap":"","mirror":"","name":"Peru","zone":"America/Lima"},"pf":{"kmap":"","mirror":"","name":"French Polynesia","zone":"Pacific/Tahiti"},"pg":{"kmap":"","mirror":"","name":"Papua New Guinea","zone":"Pacific/Port_Moresby"},"ph":{"kmap":"","mirror":"","name":"Philippines","zone":"Asia/Manila"},"pk":{"kmap":"","mirror":"","name":"Pakistan","zone":"Asia/Karachi"},"pl":{"kmap":"pl","mirror":"ftp.pl.debian.org","name":"Poland","zone":"Europe/Warsaw"},"pm":{"kmap":"","mirror":"","name":"Saint Pierre and Miquelon","zone":"America/Miquelon"},"pn":{"kmap":"","mirror":"","name":"Pitcairn","zone":"Pacific/Pitcairn"},"pr":{"kmap":"","mirror":"","name":"Puerto Rico","zone":"America/Puerto_Rico"},"ps":{"kmap":"","mirror":"","name":"Palestine, State of","zone":"Asia/Gaza"},"pt":{"kmap":"pt","mirror":"ftp.pt.debian.org","name":"Portugal","zone":"Europe/Lisbon"},"pw":{"kmap":"","mirror":"","name":"Palau","zone":"Pacific/Palau"},"py":{"kmap":"","mirror":"","name":"Paraguay","zone":"America/Asuncion"},"qa":{"kmap":"","mirror":"","name":"Qatar","zone":"Asia/Qatar"},"re":{"kmap":"","mirror":"","name":"Réunion","zone":"Indian/Reunion"},"ro":{"kmap":"","mirror":"ftp.ro.debian.org","name":"Romania","zone":"Europe/Bucharest"},"rs":{"kmap":"","mirror":"","name":"Serbia","zone":"Europe/Belgrade"},"ru":{"kmap":"","mirror":"ftp.ru.debian.org","name":"Russian Federation","zone":"Europe/Kaliningrad"},"rw":{"kmap":"","mirror":"","name":"Rwanda","zone":"Africa/Kigali"},"sa":{"kmap":"","mirror":"","name":"Saudi Arabia","zone":"Asia/Riyadh"},"sb":{"kmap":"","mirror":"","name":"Solomon Islands","zone":"Pacific/Guadalcanal"},"sc":{"kmap":"","mirror":"","name":"Seychelles","zone":"Indian/Mahe"},"sd":{"kmap":"","mirror":"","name":"Sudan","zone":"Africa/Khartoum"},"se":{"kmap":"","mirror":"ftp.se.debian.org","name":"Sweden","zone":"Europe/Stockholm"},"sg":{"kmap":"","mirror":"","name":"Singapore","zone":"Asia/Singapore"},"sh":{"kmap":"","mirror":"","name":"Saint Helena, Ascension and Tristan da Cunha","zone":"Atlantic/St_Helena"},"si":{"kmap":"si","mirror":"ftp.si.debian.org","name":"Slovenia","zone":"Europe/Ljubljana"},"sj":{"kmap":"","mirror":"","name":"Svalbard and Jan Mayen","zone":"Arctic/Longyearbyen"},"sk":{"kmap":"","mirror":"ftp.sk.debian.org","name":"Slovakia","zone":"Europe/Bratislava"},"sl":{"kmap":"","mirror":"","name":"Sierra Leone","zone":"Africa/Freetown"},"sm":{"kmap":"","mirror":"","name":"San Marino","zone":"Europe/San_Marino"},"sn":{"kmap":"","mirror":"","name":"Senegal","zone":"Africa/Dakar"},"so":{"kmap":"","mirror":"","name":"Somalia","zone":"Africa/Mogadishu"},"sr":{"kmap":"","mirror":"","name":"Suriname","zone":"America/Paramaribo"},"ss":{"kmap":"","mirror":"","name":"South Sudan","zone":"Africa/Juba"},"st":{"kmap":"","mirror":"","name":"Sao Tome and Principe","zone":"Africa/Sao_Tome"},"sv":{"kmap":"","mirror":"","name":"El Salvador","zone":"America/El_Salvador"},"sx":{"kmap":"","mirror":"","name":"Sint Maarten (Dutch part)","zone":"America/Lower_Princes"},"sy":{"kmap":"","mirror":"","name":"Syria","zone":"Asia/Damascus"},"sz":{"kmap":"","mirror":"","name":"Eswatini","zone":"Africa/Mbabane"},"tc":{"kmap":"","mirror":"","name":"Turks and Caicos Islands","zone":"America/Grand_Turk"},"td":{"kmap":"","mirror":"","name":"Chad","zone":"Africa/Ndjamena"},"tf":{"kmap":"","mirror":"","name":"French Southern Territories","zone":"Indian/Kerguelen"},"tg":{"kmap":"","mirror":"","name":"Togo","zone":"Africa/Lome"},"th":{"kmap":"","mirror":"","name":"Thailand","zone":"Asia/Bangkok"},"tj":{"kmap":"","mirror":"","name":"Tajikistan","zone":"Asia/Dushanbe"},"tk":{"kmap":"","mirror":"","name":"Tokelau","zone":"Pacific/Fakaofo"},"tl":{"kmap":"","mirror":"","name":"Timor-Leste","zone":"Asia/Dili"},"tm":{"kmap":"","mirror":"","name":"Turkmenistan","zone":"Asia/Ashgabat"},"tn":{"kmap":"","mirror":"","name":"Tunisia","zone":"Africa/Tunis"},"to":{"kmap":"","mirror":"","name":"Tonga","zone":"Pacific/Tongatapu"},"tr":{"kmap":"","mirror":"ftp.tr.debian.org","name":"Türkiye","zone":"Europe/Istanbul"},"tt":{"kmap":"","mirror":"","name":"Trinidad and Tobago","zone":"America/Port_of_Spain"},"tv":{"kmap":"","mirror":"","name":"Tuvalu","zone":"Pacific/Funafuti"},"tw":{"kmap":"","mirror":"ftp.tw.debian.org","name":"Taiwan","zone":"Asia/Taipei"},"tz":{"kmap":"","mirror":"","name":"Tanzania","zone":"Africa/Dar_es_Salaam"},"ua":{"kmap":"","mirror":"","name":"Ukraine","zone":"Europe/Simferopol"},"ug":{"kmap":"","mirror":"","name":"Uganda","zone":"Africa/Kampala"},"um":{"kmap":"","mirror":"","name":"United States Minor Outlying Islands","zone":"Pacific/Midway"},"us":{"kmap":"en-us","mirror":"ftp.us.debian.org","name":"United States","zone":"America/New_York"},"uy":{"kmap":"","mirror":"","name":"Uruguay","zone":"America/Montevideo"},"uz":{"kmap":"","mirror":"","name":"Uzbekistan","zone":"Asia/Samarkand"},"va":{"kmap":"it","mirror":"","name":"Holy See (Vatican City State)","zone":"Europe/Vatican"},"vc":{"kmap":"","mirror":"","name":"Saint Vincent and the Grenadines","zone":"America/St_Vincent"},"ve":{"kmap":"","mirror":"","name":"Venezuela","zone":"America/Caracas"},"vg":{"kmap":"","mirror":"","name":"Virgin Islands, British","zone":"America/Tortola"},"vi":{"kmap":"","mirror":"","name":"Virgin Islands, U.S.","zone":"America/St_Thomas"},"vn":{"kmap":"","mirror":"","name":"Vietnam","zone":"Asia/Ho_Chi_Minh"},"vu":{"kmap":"","mirror":"","name":"Vanuatu","zone":"Pacific/Efate"},"wf":{"kmap":"","mirror":"","name":"Wallis and Futuna","zone":"Pacific/Wallis"},"ws":{"kmap":"","mirror":"","name":"Samoa","zone":"Pacific/Apia"},"ye":{"kmap":"","mirror":"","name":"Yemen","zone":"Asia/Aden"},"yt":{"kmap":"","mirror":"","name":"Mayotte","zone":"Indian/Mayotte"},"za":{"kmap":"","mirror":"","name":"South Africa","zone":"Africa/Johannesburg"},"zm":{"kmap":"","mirror":"","name":"Zambia","zone":"Africa/Lusaka"},"zw":{"kmap":"","mirror":"","name":"Zimbabwe","zone":"Africa/Harare"}},"countryhash":{"afghanistan":"af","albania":"al","algeria":"dz","american samoa":"as","andorra":"ad","angola":"ao","anguilla":"ai","antarctica":"aq","antigua and barbuda":"ag","argentina":"ar","armenia":"am","aruba":"aw","australia":"au","austria":"at","azerbaijan":"az","bahamas":"bs","bahrain":"bh","bangladesh":"bd","barbados":"bb","belarus":"by","belgium":"be","belize":"bz","benin":"bj","bermuda":"bm","bhutan":"bt","bolivia":"bo","bonaire, sint eustatius and saba":"bq","bosnia and herzegovina":"ba","botswana":"bw","bouvet island":"bv","brazil":"br","british indian ocean territory":"io","brunei darussalam":"bn","bulgaria":"bg","burkina faso":"bf","burundi":"bi","cabo verde":"cv","cambodia":"kh","cameroon":"cm","canada":"ca","cayman islands":"ky","central african republic":"cf","chad":"td","chile":"cl","china":"cn","christmas island":"cx","cocos (keeling) islands":"cc","colombia":"co","comoros":"km","congo":"cg","congo, the democratic republic of the":"cd","cook islands":"ck","costa rica":"cr","croatia":"hr","cuba":"cu","curaçao":"cw","cyprus":"cy","czechia":"cz","côte d'ivoire":"ci","denmark":"dk","djibouti":"dj","dominica":"dm","dominican republic":"do","ecuador":"ec","egypt":"eg","el salvador":"sv","equatorial guinea":"gq","eritrea":"er","estonia":"ee","eswatini":"sz","ethiopia":"et","falkland islands (malvinas)":"fk","faroe islands":"fo","fiji":"fj","finland":"fi","france":"fr","french guiana":"gf","french polynesia":"pf","french southern territories":"tf","gabon":"ga","gambia":"gm","georgia":"ge","germany":"de","ghana":"gh","gibraltar":"gi","greece":"gr","greenland":"gl","grenada":"gd","guadeloupe":"gp","guam":"gu","guatemala":"gt","guernsey":"gg","guinea":"gn","guinea-bissau":"gw","guyana":"gy","haiti":"ht","heard island and mcdonald islands":"hm","holy see (vatican city state)":"va","honduras":"hn","hong kong":"hk","hungary":"hu","iceland":"is","india":"in","indonesia":"id","iran":"ir","iraq":"iq","ireland":"ie","isle of man":"im","israel":"il","italy":"it","jamaica":"jm","japan":"jp","jersey":"je","jordan":"jo","kazakhstan":"kz","kenya":"ke","kiribati":"ki","kuwait":"kw","kyrgyzstan":"kg","laos":"la","latvia":"lv","lebanon":"lb","lesotho":"ls","liberia":"lr","libya":"ly","liechtenstein":"li","lithuania":"lt","luxembourg":"lu","macao":"mo","madagascar":"mg","malawi":"mw","malaysia":"my","maldives":"mv","mali":"ml","malta":"mt","marshall islands":"mh","martinique":"mq","mauritania":"mr","mauritius":"mu","mayotte":"yt","mexico":"mx","micronesia, federated states of":"fm","moldova":"md","monaco":"mc","mongolia":"mn","montenegro":"me","montserrat":"ms","morocco":"ma","mozambique":"mz","myanmar":"mm","namibia":"na","nauru":"nr","nepal":"np","netherlands":"nl","new caledonia":"nc","new zealand":"nz","nicaragua":"ni","niger":"ne","nigeria":"ng","niue":"nu","norfolk island":"nf","north korea":"kp","north macedonia":"mk","northern mariana islands":"mp","norway":"no","oman":"om","pakistan":"pk","palau":"pw","palestine, state of":"ps","panama":"pa","papua new guinea":"pg","paraguay":"py","peru":"pe","philippines":"ph","pitcairn":"pn","poland":"pl","portugal":"pt","puerto rico":"pr","qatar":"qa","romania":"ro","russian federation":"ru","rwanda":"rw","réunion":"re","saint barthélemy":"bl","saint helena, ascension and tristan da cunha":"sh","saint kitts and nevis":"kn","saint lucia":"lc","saint martin (french part)":"mf","saint pierre and miquelon":"pm","saint vincent and the grenadines":"vc","samoa":"ws","san marino":"sm","sao tome and principe":"st","saudi arabia":"sa","senegal":"sn","serbia":"rs","seychelles":"sc","sierra leone":"sl","singapore":"sg","sint maarten (dutch part)":"sx","slovakia":"sk","slovenia":"si","solomon islands":"sb","somalia":"so","south africa":"za","south georgia and the south sandwich islands":"gs","south korea":"kr","south sudan":"ss","spain":"es","sri lanka":"lk","sudan":"sd","suriname":"sr","svalbard and jan mayen":"sj","sweden":"se","switzerland":"ch","syria":"sy","taiwan":"tw","tajikistan":"tj","tanzania":"tz","thailand":"th","timor-leste":"tl","togo":"tg","tokelau":"tk","tonga":"to","trinidad and tobago":"tt","tunisia":"tn","turkmenistan":"tm","turks and caicos islands":"tc","tuvalu":"tv","türkiye":"tr","uganda":"ug","ukraine":"ua","united arab emirates":"ae","united kingdom":"gb","united states":"us","united states minor outlying islands":"um","uruguay":"uy","uzbekistan":"uz","vanuatu":"vu","venezuela":"ve","vietnam":"vn","virgin islands, british":"vg","virgin islands, u.s.":"vi","wallis and futuna":"wf","western sahara":"eh","yemen":"ye","zambia":"zm","zimbabwe":"zw","åland islands":"ax"},"kmap":{"de":{"console":"qwertz/de-latin1-nodeadkeys.kmap.gz","kvm":"de","name":"German","x11":"de","x11var":"nodeadkeys"},"de-ch":{"console":"qwertz/sg-latin1.kmap.gz","kvm":"de-ch","name":"Swiss-German","x11":"ch","x11var":"de_nodeadkeys"},"dk":{"console":"qwerty/dk-latin1.kmap.gz","kvm":"da","name":"Danish","x11":"dk","x11var":"nodeadkeys"},"en-gb":{"console":"qwerty/uk.kmap.gz","kvm":"en-gb","name":"United Kingdom","x11":"gb","x11var":""},"en-us":{"console":"qwerty/us-latin1.kmap.gz","kvm":"en-us","name":"U.S. English","x11":"us","x11var":""},"es":{"console":"qwerty/es.kmap.gz","kvm":"es","name":"Spanish","x11":"es","x11var":"nodeadkeys"},"fi":{"console":"qwerty/fi-latin1.kmap.gz","kvm":"fi","name":"Finnish","x11":"fi","x11var":"nodeadkeys"},"fr":{"console":"azerty/fr-latin1.kmap.gz","kvm":"fr","name":"French","x11":"fr","x11var":"nodeadkeys"},"fr-be":{"console":"azerty/be2-latin1.kmap.gz","kvm":"fr-be","name":"Belgium-French","x11":"be","x11var":"nodeadkeys"},"fr-ca":{"console":"qwerty/cf.kmap.gz","kvm":"fr-ca","name":"Canada-French","x11":"ca","x11var":"fr-legacy"},"fr-ch":{"console":"qwertz/fr_CH-latin1.kmap.gz","kvm":"fr-ch","name":"Swiss-French","x11":"ch","x11var":"fr_nodeadkeys"},"hu":{"console":"qwertz/hu.kmap.gz","kvm":"hu","name":"Hungarian","x11":"hu","x11var":""},"is":{"console":"qwerty/is-latin1.kmap.gz","kvm":"is","name":"Icelandic","x11":"is","x11var":"nodeadkeys"},"it":{"console":"qwerty/it2.kmap.gz","kvm":"it","name":"Italian","x11":"it","x11var":"nodeadkeys"},"jp":{"console":"qwerty/jp106.kmap.gz","kvm":"ja","name":"Japanese","x11":"jp","x11var":""},"lt":{"console":"qwerty/lt.kmap.gz","kvm":"lt","name":"Lithuanian","x11":"lt","x11var":"std"},"mk":{"console":"qwerty/mk.kmap.gz","kvm":"mk","name":"Macedonian","x11":"mk","x11var":"nodeadkeys"},"nl":{"console":"qwerty/nl.kmap.gz","kvm":"nl","name":"Dutch","x11":"nl","x11var":""},"no":{"console":"qwerty/no-latin1.kmap.gz","kvm":"no","name":"Norwegian","x11":"no","x11var":"nodeadkeys"},"pl":{"console":"qwerty/pl.kmap.gz","kvm":"pl","name":"Polish","x11":"pl","x11var":""},"pt":{"console":"qwerty/pt-latin1.kmap.gz","kvm":"pt","name":"Portuguese","x11":"pt","x11var":"nodeadkeys"},"pt-br":{"console":"qwerty/br-latin1.kmap.gz","kvm":"pt-br","name":"Brazil-Portuguese","x11":"br","x11var":"nodeadkeys"},"se":{"console":"qwerty/se-latin1.kmap.gz","kvm":"sv","name":"Swedish","x11":"se","x11var":"nodeadkeys"},"si":{"console":"qwertz/slovene.kmap.gz","kvm":"sl","name":"Slovenian","x11":"si","x11var":""},"tr":{"console":"qwerty/trq.kmap.gz","kvm":"tr","name":"Turkish","x11":"tr","x11var":""}},"kmaphash":{"Belgium-French":"fr-be","Brazil-Portuguese":"pt-br","Canada-French":"fr-ca","Danish":"dk","Dutch":"nl","Finnish":"fi","French":"fr","German":"de","Hungarian":"hu","Icelandic":"is","Italian":"it","Japanese":"jp","Lithuanian":"lt","Macedonian":"mk","Norwegian":"no","Polish":"pl","Portuguese":"pt","Slovenian":"si","Spanish":"es","Swedish":"se","Swiss-French":"fr-ch","Swiss-German":"de-ch","Turkish":"tr","U.S. English":"en-us","United Kingdom":"en-gb"},"zones":{"Africa/Abidjan":1,"Africa/Accra":1,"Africa/Addis_Ababa":1,"Africa/Algiers":1,"Africa/Asmara":1,"Africa/Bamako":1,"Africa/Bangui":1,"Africa/Banjul":1,"Africa/Bissau":1,"Africa/Blantyre":1,"Africa/Brazzaville":1,"Africa/Bujumbura":1,"Africa/Cairo":1,"Africa/Casablanca":1,"Africa/Ceuta":1,"Africa/Conakry":1,"Africa/Dakar":1,"Africa/Dar_es_Salaam":1,"Africa/Djibouti":1,"Africa/Douala":1,"Africa/El_Aaiun":1,"Africa/Freetown":1,"Africa/Gaborone":1,"Africa/Harare":1,"Africa/Johannesburg":1,"Africa/Juba":1,"Africa/Kampala":1,"Africa/Khartoum":1,"Africa/Kigali":1,"Africa/Kinshasa":1,"Africa/Lagos":1,"Africa/Libreville":1,"Africa/Lome":1,"Africa/Luanda":1,"Africa/Lubumbashi":1,"Africa/Lusaka":1,"Africa/Malabo":1,"Africa/Maputo":1,"Africa/Maseru":1,"Africa/Mbabane":1,"Africa/Mogadishu":1,"Africa/Monrovia":1,"Africa/Nairobi":1,"Africa/Ndjamena":1,"Africa/Niamey":1,"Africa/Nouakchott":1,"Africa/Ouagadougou":1,"Africa/Porto-Novo":1,"Africa/Sao_Tome":1,"Africa/Tripoli":1,"Africa/Tunis":1,"Africa/Windhoek":1,"America/Adak":1,"America/Anchorage":1,"America/Anguilla":1,"America/Antigua":1,"America/Araguaina":1,"America/Argentina/Buenos_Aires":1,"America/Argentina/Catamarca":1,"America/Argentina/Cordoba":1,"America/Argentina/Jujuy":1,"America/Argentina/La_Rioja":1,"America/Argentina/Mendoza":1,"America/Argentina/Rio_Gallegos":1,"America/Argentina/Salta":1,"America/Argentina/San_Juan":1,"America/Argentina/San_Luis":1,"America/Argentina/Tucuman":1,"America/Argentina/Ushuaia":1,"America/Aruba":1,"America/Asuncion":1,"America/Atikokan":1,"America/Bahia":1,"America/Bahia_Banderas":1,"America/Barbados":1,"America/Belem":1,"America/Belize":1,"America/Blanc-Sablon":1,"America/Boa_Vista":1,"America/Bogota":1,"America/Boise":1,"America/Cambridge_Bay":1,"America/Campo_Grande":1,"America/Cancun":1,"America/Caracas":1,"America/Cayenne":1,"America/Cayman":1,"America/Chicago":1,"America/Chihuahua":1,"America/Ciudad_Juarez":1,"America/Costa_Rica":1,"America/Creston":1,"America/Cuiaba":1,"America/Curacao":1,"America/Danmarkshavn":1,"America/Dawson":1,"America/Dawson_Creek":1,"America/Denver":1,"America/Detroit":1,"America/Dominica":1,"America/Edmonton":1,"America/Eirunepe":1,"America/El_Salvador":1,"America/Fort_Nelson":1,"America/Fortaleza":1,"America/Glace_Bay":1,"America/Goose_Bay":1,"America/Grand_Turk":1,"America/Grenada":1,"America/Guadeloupe":1,"America/Guatemala":1,"America/Guayaquil":1,"America/Guyana":1,"America/Halifax":1,"America/Havana":1,"America/Hermosillo":1,"America/Indiana/Indianapolis":1,"America/Indiana/Knox":1,"America/Indiana/Marengo":1,"America/Indiana/Petersburg":1,"America/Indiana/Tell_City":1,"America/Indiana/Vevay":1,"America/Indiana/Vincennes":1,"America/Indiana/Winamac":1,"America/Inuvik":1,"America/Iqaluit":1,"America/Jamaica":1,"America/Juneau":1,"America/Kentucky/Louisville":1,"America/Kentucky/Monticello":1,"America/Kralendijk":1,"America/La_Paz":1,"America/Lima":1,"America/Los_Angeles":1,"America/Lower_Princes":1,"America/Maceio":1,"America/Managua":1,"America/Manaus":1,"America/Marigot":1,"America/Martinique":1,"America/Matamoros":1,"America/Mazatlan":1,"America/Menominee":1,"America/Merida":1,"America/Metlakatla":1,"America/Mexico_City":1,"America/Miquelon":1,"America/Moncton":1,"America/Monterrey":1,"America/Montevideo":1,"America/Montserrat":1,"America/Nassau":1,"America/New_York":1,"America/Nome":1,"America/Noronha":1,"America/North_Dakota/Beulah":1,"America/North_Dakota/Center":1,"America/North_Dakota/New_Salem":1,"America/Nuuk":1,"America/Ojinaga":1,"America/Panama":1,"America/Paramaribo":1,"America/Phoenix":1,"America/Port-au-Prince":1,"America/Port_of_Spain":1,"America/Porto_Velho":1,"America/Puerto_Rico":1,"America/Punta_Arenas":1,"America/Rankin_Inlet":1,"America/Recife":1,"America/Regina":1,"America/Resolute":1,"America/Rio_Branco":1,"America/Santarem":1,"America/Santiago":1,"America/Santo_Domingo":1,"America/Sao_Paulo":1,"America/Scoresbysund":1,"America/Sitka":1,"America/St_Barthelemy":1,"America/St_Johns":1,"America/St_Kitts":1,"America/St_Lucia":1,"America/St_Thomas":1,"America/St_Vincent":1,"America/Swift_Current":1,"America/Tegucigalpa":1,"America/Thule":1,"America/Tijuana":1,"America/Toronto":1,"America/Tortola":1,"America/Vancouver":1,"America/Whitehorse":1,"America/Winnipeg":1,"America/Yakutat":1,"Antarctica/Casey":1,"Antarctica/Davis":1,"Antarctica/DumontDUrville":1,"Antarctica/Macquarie":1,"Antarctica/Mawson":1,"Antarctica/McMurdo":1,"Antarctica/Palmer":1,"Antarctica/Rothera":1,"Antarctica/Syowa":1,"Antarctica/Troll":1,"Antarctica/Vostok":1,"Arctic/Longyearbyen":1,"Asia/Aden":1,"Asia/Almaty":1,"Asia/Amman":1,"Asia/Anadyr":1,"Asia/Aqtau":1,"Asia/Aqtobe":1,"Asia/Ashgabat":1,"Asia/Atyrau":1,"Asia/Baghdad":1,"Asia/Bahrain":1,"Asia/Baku":1,"Asia/Bangkok":1,"Asia/Barnaul":1,"Asia/Beirut":1,"Asia/Bishkek":1,"Asia/Brunei":1,"Asia/Chita":1,"Asia/Choibalsan":1,"Asia/Colombo":1,"Asia/Damascus":1,"Asia/Dhaka":1,"Asia/Dili":1,"Asia/Dubai":1,"Asia/Dushanbe":1,"Asia/Famagusta":1,"Asia/Gaza":1,"Asia/Hebron":1,"Asia/Ho_Chi_Minh":1,"Asia/Hong_Kong":1,"Asia/Hovd":1,"Asia/Irkutsk":1,"Asia/Jakarta":1,"Asia/Jayapura":1,"Asia/Jerusalem":1,"Asia/Kabul":1,"Asia/Kamchatka":1,"Asia/Karachi":1,"Asia/Kathmandu":1,"Asia/Khandyga":1,"Asia/Kolkata":1,"Asia/Krasnoyarsk":1,"Asia/Kuala_Lumpur":1,"Asia/Kuching":1,"Asia/Kuwait":1,"Asia/Macau":1,"Asia/Magadan":1,"Asia/Makassar":1,"Asia/Manila":1,"Asia/Muscat":1,"Asia/Nicosia":1,"Asia/Novokuznetsk":1,"Asia/Novosibirsk":1,"Asia/Omsk":1,"Asia/Oral":1,"Asia/Phnom_Penh":1,"Asia/Pontianak":1,"Asia/Pyongyang":1,"Asia/Qatar":1,"Asia/Qostanay":1,"Asia/Qyzylorda":1,"Asia/Riyadh":1,"Asia/Sakhalin":1,"Asia/Samarkand":1,"Asia/Seoul":1,"Asia/Shanghai":1,"Asia/Singapore":1,"Asia/Srednekolymsk":1,"Asia/Taipei":1,"Asia/Tashkent":1,"Asia/Tbilisi":1,"Asia/Tehran":1,"Asia/Thimphu":1,"Asia/Tokyo":1,"Asia/Tomsk":1,"Asia/Ulaanbaatar":1,"Asia/Urumqi":1,"Asia/Ust-Nera":1,"Asia/Vientiane":1,"Asia/Vladivostok":1,"Asia/Yakutsk":1,"Asia/Yangon":1,"Asia/Yekaterinburg":1,"Asia/Yerevan":1,"Atlantic/Azores":1,"Atlantic/Bermuda":1,"Atlantic/Canary":1,"Atlantic/Cape_Verde":1,"Atlantic/Faroe":1,"Atlantic/Madeira":1,"Atlantic/Reykjavik":1,"Atlantic/South_Georgia":1,"Atlantic/St_Helena":1,"Atlantic/Stanley":1,"Australia/Adelaide":1,"Australia/Brisbane":1,"Australia/Broken_Hill":1,"Australia/Darwin":1,"Australia/Eucla":1,"Australia/Hobart":1,"Australia/Lindeman":1,"Australia/Lord_Howe":1,"Australia/Melbourne":1,"Australia/Perth":1,"Australia/Sydney":1,"Europe/Amsterdam":1,"Europe/Andorra":1,"Europe/Astrakhan":1,"Europe/Athens":1,"Europe/Belgrade":1,"Europe/Berlin":1,"Europe/Bratislava":1,"Europe/Brussels":1,"Europe/Bucharest":1,"Europe/Budapest":1,"Europe/Busingen":1,"Europe/Chisinau":1,"Europe/Copenhagen":1,"Europe/Dublin":1,"Europe/Gibraltar":1,"Europe/Guernsey":1,"Europe/Helsinki":1,"Europe/Isle_of_Man":1,"Europe/Istanbul":1,"Europe/Jersey":1,"Europe/Kaliningrad":1,"Europe/Kirov":1,"Europe/Kyiv":1,"Europe/Lisbon":1,"Europe/Ljubljana":1,"Europe/London":1,"Europe/Luxembourg":1,"Europe/Madrid":1,"Europe/Malta":1,"Europe/Mariehamn":1,"Europe/Minsk":1,"Europe/Monaco":1,"Europe/Moscow":1,"Europe/Oslo":1,"Europe/Paris":1,"Europe/Podgorica":1,"Europe/Prague":1,"Europe/Riga":1,"Europe/Rome":1,"Europe/Samara":1,"Europe/San_Marino":1,"Europe/Sarajevo":1,"Europe/Saratov":1,"Europe/Simferopol":1,"Europe/Skopje":1,"Europe/Sofia":1,"Europe/Stockholm":1,"Europe/Tallinn":1,"Europe/Tirane":1,"Europe/Ulyanovsk":1,"Europe/Vaduz":1,"Europe/Vatican":1,"Europe/Vienna":1,"Europe/Vilnius":1,"Europe/Volgograd":1,"Europe/Warsaw":1,"Europe/Zagreb":1,"Europe/Zurich":1,"Indian/Antananarivo":1,"Indian/Chagos":1,"Indian/Christmas":1,"Indian/Cocos":1,"Indian/Comoro":1,"Indian/Kerguelen":1,"Indian/Mahe":1,"Indian/Maldives":1,"Indian/Mauritius":1,"Indian/Mayotte":1,"Indian/Reunion":1,"Pacific/Apia":1,"Pacific/Auckland":1,"Pacific/Bougainville":1,"Pacific/Chatham":1,"Pacific/Chuuk":1,"Pacific/Easter":1,"Pacific/Efate":1,"Pacific/Fakaofo":1,"Pacific/Fiji":1,"Pacific/Funafuti":1,"Pacific/Galapagos":1,"Pacific/Gambier":1,"Pacific/Guadalcanal":1,"Pacific/Guam":1,"Pacific/Honolulu":1,"Pacific/Kanton":1,"Pacific/Kiritimati":1,"Pacific/Kosrae":1,"Pacific/Kwajalein":1,"Pacific/Majuro":1,"Pacific/Marquesas":1,"Pacific/Midway":1,"Pacific/Nauru":1,"Pacific/Niue":1,"Pacific/Norfolk":1,"Pacific/Noumea":1,"Pacific/Pago_Pago":1,"Pacific/Palau":1,"Pacific/Pitcairn":1,"Pacific/Pohnpei":1,"Pacific/Port_Moresby":1,"Pacific/Rarotonga":1,"Pacific/Saipan":1,"Pacific/Tahiti":1,"Pacific/Tarawa":1,"Pacific/Tongatapu":1,"Pacific/Wake":1,"Pacific/Wallis":1}}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
new file mode 100644
index 0000000..f228b1e
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
@@ -0,0 +1,29 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "6": "6",
+ "7": "7",
+ "8": "8",
+ "9": "9"
+ },
+ "filesys": "zfs (RAID10)",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
new file mode 100644
index 0000000..458351e
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid10"
+#disk_list = ['sda']
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
new file mode 100644
index 0000000..2990922
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
@@ -0,0 +1,26 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "9": "9"
+ },
+ "filesys": "zfs (RAID0)",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
new file mode 100644
index 0000000..ed05f89
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid0"
+filter_match = "all"
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+filter.ID_SERIAL_SHORT = "S2HRNX0J403419"
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
new file mode 100644
index 0000000..8c22300
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
@@ -0,0 +1,33 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "0": "0",
+ "1": "1",
+ "2": "2",
+ "3": "3",
+ "6": "6",
+ "7": "7",
+ "8": "8",
+ "9": "9"
+ },
+ "filesys": "zfs (RAID10)",
+ "gateway": "192.168.1.1",
+ "hdsize": 2980.820640563965,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "on",
+ "copies": 1
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
new file mode 100644
index 0000000..420fb40
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
@@ -0,0 +1,17 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid10"
+filter_match = "any"
+filter.ID_SERIAL = "*MZ7KM240HAGR*"
+filter.ID_MODEL = "Micron_9300*"
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.json b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json
new file mode 100644
index 0000000..9021377
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "192.168.1.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
new file mode 100644
index 0000000..4900ca5
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
@@ -0,0 +1,14 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
new file mode 100644
index 0000000..6f31079
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "10.10.10.10/24",
+ "country": "at",
+ "dns": "10.10.10.1",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "10.10.10.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "enp65s0f0",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
new file mode 100644
index 0000000..37e8a22
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
@@ -0,0 +1,19 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = false
+cidr = "10.10.10.10/24"
+dns = "10.10.10.1"
+gateway = "10.10.10.1"
+filter.ID_NET_NAME_MAC = "*a0369f0ab382"
+
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/readme b/proxmox-auto-installer/tests/resources/parse_answer/readme
new file mode 100644
index 0000000..6ce77ae
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/readme
@@ -0,0 +1,4 @@
+the size parameter from /sys/block/{disk}/size is the number of blocks.
+
+to calculate the size as the low level installer needs it:
+size * 512 / 1024 / 1024 / 1024 with 14 digits after the comma
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
new file mode 100644
index 0000000..515cc89
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
@@ -0,0 +1,17 @@
+{
+ "autoreboot": 1,
+ "cidr": "10.10.10.10/24",
+ "country": "at",
+ "dns": "10.10.10.1",
+ "domain": "testinstall",
+ "filesys": "ext4",
+ "gateway": "10.10.10.1",
+ "hdsize": 223.57088470458984,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "enp129s0f1np1",
+ "password": "123456",
+ "target_hd": "/dev/sda",
+ "timezone": "Europe/Vienna"
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
new file mode 100644
index 0000000..b7d7728
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
@@ -0,0 +1,19 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = false
+cidr = "10.10.10.10/24"
+dns = "10.10.10.1"
+gateway = "10.10.10.1"
+filter.ID_NET_NAME = "enp129s0f1np1"
+
+
+[disk-setup]
+filesystem = "ext4"
+disk_list = ["sda"]
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.json b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json
new file mode 100644
index 0000000..c29a303
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.json
@@ -0,0 +1,27 @@
+{
+ "autoreboot": 1,
+ "cidr": "192.168.1.114/24",
+ "country": "at",
+ "dns": "192.168.1.254",
+ "domain": "testinstall",
+ "disk_selection": {
+ "6": "6",
+ "7": "7"
+ },
+ "filesys": "zfs (RAID1)",
+ "gateway": "192.168.1.1",
+ "hdsize": 80.0,
+ "hostname": "pveauto",
+ "keymap": "de",
+ "mailto": "mail@no.invalid",
+ "mngmt_nic": "eno1",
+ "password": "123456",
+ "timezone": "Europe/Vienna",
+ "zfs_opts": {
+ "arc_max": 2048,
+ "ashift": 12,
+ "checksum": "on",
+ "compress": "lz4",
+ "copies": 2
+ }
+}
diff --git a/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
new file mode 100644
index 0000000..5410ed6
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
@@ -0,0 +1,20 @@
+[global]
+keyboard = "de"
+country = "at"
+fqdn = "pveauto.testinstall"
+mailto = "mail@no.invalid"
+timezone = "Europe/Vienna"
+password = "123456"
+
+[network]
+use_dhcp = true
+
+[disk-setup]
+filesystem = "zfs"
+zfs.raid = "raid1"
+zfs.ashift = 12
+zfs.checksum = "on"
+zfs.compress = "lz4"
+zfs.copies = 2
+zfs.hdsize = 80
+disk_list = ["sda", "sdb"]
diff --git a/proxmox-auto-installer/tests/resources/run-env-info.json b/proxmox-auto-installer/tests/resources/run-env-info.json
new file mode 100644
index 0000000..6762470
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/run-env-info.json
@@ -0,0 +1 @@
+{"boot_type":"efi","country":"at","disks":[[0,"/dev/nvme0n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme0n1"],[1,"/dev/nvme1n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme1n1"],[2,"/dev/nvme2n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme2n1"],[3,"/dev/nvme3n1",6251233968,"Micron_9300_MTFDHAL3T2TDR",4096,"/sys/block/nvme3n1"],[4,"/dev/nvme4n1",976773168,"Samsung SSD 970 EVO Plus 500GB",512,"/sys/block/nvme4n1"],[5,"/dev/nvme5n1",732585168,"INTEL SSDPED1K375GA",512,"/sys/block/nvme5n1"],[6,"/dev/sda",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sda"],[7,"/dev/sdb",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdb"],[8,"/dev/sdc",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdc"],[9,"/dev/sdd",468862128,"SAMSUNG MZ7KM240",512,"/sys/block/sdd"]],"hvm_supported":1,"ipconf":{"default":"4","dnsserver":"192.168.1.254","domain":null,"gateway":"192.168.1.1","ifaces":{"10":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"2":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"3":{"driver":"igb","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"4":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","inet":{"addr":"192.168.1.114","mask":"255.255.240.0","prefix":20},"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"5":{"driver":"cdc_ether","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"},"6":{"driver":"igb","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"7":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"8":{"driver":"mlx5_core","flags":"NO-CARRIER,BROADCAST,MULTICAST,UP","mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"9":{"driver":"mlx5_core","flags":"BROADCAST,MULTICAST,UP,LOWER_UP","mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"}}},"kernel_cmdline":"BOOT_IMAGE=/boot/linux26 ro ramdisk_size=16777216 rw splash=verbose proxdebug vga=788","network":{"dns":{"dns":["192.168.1.254"],"domain":null},"interfaces":{"eno1":{"addresses":[{"address":"192.168.1.114","family":"inet","prefix":24}],"index":4,"mac":"b4:2e:99:ac:ad:b4","name":"eno1","state":"UP"},"eno2":{"index":6,"mac":"b4:2e:99:ac:ad:b5","name":"eno2","state":"UP"},"enp129s0f0np0":{"index":7,"mac":"1c:34:da:5c:5e:24","name":"enp129s0f0np0","state":"DOWN"},"enp129s0f1np1":{"index":8,"mac":"1c:34:da:5c:5e:25","name":"enp129s0f1np1","state":"DOWN"},"enp193s0f0np0":{"index":9,"mac":"24:8a:07:1e:05:bc","name":"enp193s0f0np0","state":"UP"},"enp193s0f1np1":{"index":10,"mac":"24:8a:07:1e:05:bd","name":"enp193s0f1np1","state":"DOWN"},"enp65s0f0":{"index":2,"mac":"a0:36:9f:0a:b3:82","name":"enp65s0f0","state":"DOWN"},"enp65s0f1":{"index":3,"mac":"a0:36:9f:0a:b3:83","name":"enp65s0f1","state":"DOWN"},"enx5a4732ddc747":{"index":5,"mac":"5a:47:32:dd:c7:47","name":"enx5a4732ddc747","state":"UNKNOWN"}},"routes":{"gateway4":{"dev":"eno1","gateway":"192.168.1.1"}}},"total_memory":257597}
diff --git a/proxmox-auto-installer/tests/resources/run-env-udev.json b/proxmox-auto-installer/tests/resources/run-env-udev.json
new file mode 100644
index 0000000..4fe1f30
--- /dev/null
+++ b/proxmox-auto-installer/tests/resources/run-env-udev.json
@@ -0,0 +1 @@
+{"disks":{"0":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:01:00.0-nvme-1 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_19502596FC74 /dev/disk/by-id/lvm-pv-uuid-hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP /dev/disk/by-diskseq/16 /dev/disk/by-id/nvme-eui.000000000000001500a075012596fc74","DEVNAME":"/dev/nvme0n1","DEVPATH":"/devices/pci0000:00/0000:00:01.1/0000:01:00.0/nvme/nvme0/nvme0n1","DEVTYPE":"disk","DISKSEQ":"16","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_UUID_ENC":"hl5Cyv-dghE-CcX8-lDCV-6BSj-EbFU-cT4dIP","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:01:00.0-nvme-1","ID_PATH_TAG":"pci-0000_01_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_19502596FC74","ID_SERIAL_SHORT":"19502596FC74","ID_WWN":"eui.000000000000001500a075012596fc74","LVM_VG_NAME_COMPLETE":"ceph-67f6a633-8bac-4ba6-a54c-40f0d24a9701","MAJOR":"259","MINOR":"6","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215609"},"1":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:02:00.0-nvme-1 /dev/disk/by-id/nvme-eui.000000000000001400a0750125de7a16 /dev/disk/by-diskseq/15 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_195225DE7A16","DEVNAME":"/dev/nvme1n1","DEVPATH":"/devices/pci0000:00/0000:00:01.2/0000:02:00.0/nvme/nvme1/nvme1n1","DEVTYPE":"disk","DISKSEQ":"15","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:02:00.0-nvme-1","ID_PATH_TAG":"pci-0000_02_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_195225DE7A16","ID_SERIAL_SHORT":"195225DE7A16","ID_WWN":"eui.000000000000001400a0750125de7a16","MAJOR":"259","MINOR":"5","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271971"},"2":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:03:00.0-nvme-1 /dev/disk/by-diskseq/17 /dev/disk/by-id/lvm-pv-uuid-b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F206E /dev/disk/by-id/nvme-eui.000000000000001400a07501250f206e","DEVNAME":"/dev/nvme2n1","DEVPATH":"/devices/pci0000:00/0000:00:01.3/0000:03:00.0/nvme/nvme2/nvme2n1","DEVTYPE":"disk","DISKSEQ":"17","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_UUID_ENC":"b92FQw-lExM-2EYR-5UyV-T6cl-yzsM-qRjCOU","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:03:00.0-nvme-1","ID_PATH_TAG":"pci-0000_03_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F206E","ID_SERIAL_SHORT":"1945250F206E","ID_WWN":"eui.000000000000001400a07501250f206e","LVM_VG_NAME_COMPLETE":"ceph-ee820014-6121-458b-a661-889f0901bff6","MAJOR":"259","MINOR":"7","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45218640"},"3":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-path/pci-0000:04:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T2TDR_1945250F20AC /dev/disk/by-diskseq/18 /dev/disk/by-id/nvme-eui.000000000000001400a07501250f20ac","DEVNAME":"/dev/nvme3n1","DEVPATH":"/devices/pci0000:00/0000:00:01.4/0000:04:00.0/nvme/nvme3/nvme3n1","DEVTYPE":"disk","DISKSEQ":"18","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_UUID_ENC":"f56spY-IptZ-fH5e-AqQv-K1cI-3nnt-2UlO17","ID_FS_VERSION":"LVM2 001","ID_MODEL":"Micron_9300_MTFDHAL3T2TDR","ID_PATH":"pci-0000:04:00.0-nvme-1","ID_PATH_TAG":"pci-0000_04_00_0-nvme-1","ID_REVISION":"11300DN0","ID_SERIAL":"Micron_9300_MTFDHAL3T2TDR_1945250F20AC","ID_SERIAL_SHORT":"1945250F20AC","ID_WWN":"eui.000000000000001400a07501250f20ac","LVM_VG_NAME_COMPLETE":"ceph-2928aceb-9300-4175-8640-e227d897d45e","MAJOR":"259","MINOR":"8","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45215244"},"4":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/13 /dev/disk/by-path/pci-0000:82:00.0-nvme-1 /dev/disk/by-id/lvm-pv-uuid-jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd /dev/disk/by-id/nvme-INTEL_SSDPED1K375GA_PHKS746500DK375AGN /dev/disk/by-id/nvme-nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","DEVNAME":"/dev/nvme4n1","DEVPATH":"/devices/pci0000:80/0000:80:03.1/0000:82:00.0/nvme/nvme4/nvme4n1","DEVTYPE":"disk","DISKSEQ":"13","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_UUID_ENC":"jFM6eE-KUmT-fTBO-9SWe-4VJG-W4rW-DUQPRd","ID_FS_VERSION":"LVM2 001","ID_MODEL":"INTEL SSDPED1K375GA","ID_PATH":"pci-0000:82:00.0-nvme-1","ID_PATH_TAG":"pci-0000_82_00_0-nvme-1","ID_REVISION":"E2010435","ID_SERIAL":"INTEL_SSDPED1K375GA_PHKS746500DK375AGN","ID_SERIAL_SHORT":"PHKS746500DK375AGN","ID_WWN":"nvme.8086-50484b53373436353030444b33373541474e-494e54454c20535344504544314b3337354741-00000001","LVM_VG_NAME_COMPLETE":"ceph-b4af8112-88e7-4cd4-9cf9-0f4163ca77bd","MAJOR":"259","MINOR":"0","SUBSYSTEM":"block","SYSTEMD_READY":"1","TAGS":":systemd:","USEC_INITIALIZED":"45219471"},"5":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/nvme-eui.0025385791b04175 /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N /dev/disk/by-path/pci-0000:06:00.0-nvme-1 /dev/disk/by-diskseq/14","DEVNAME":"/dev/nvme5n1","DEVPATH":"/devices/pci0000:00/0000:00:03.3/0000:06:00.0/nvme/nvme5/nvme5n1","DEVTYPE":"disk","DISKSEQ":"14","ID_MODEL":"Samsung SSD 970 EVO Plus 500GB","ID_PART_TABLE_TYPE":"gpt","ID_PART_TABLE_UUID":"1c40cb4b-72d8-49ec-804b-e5933e09423d","ID_PATH":"pci-0000:06:00.0-nvme-1","ID_PATH_TAG":"pci-0000_06_00_0-nvme-1","ID_REVISION":"2B2QEXM7","ID_SERIAL":"Samsung_SSD_970_EVO_Plus_500GB_S4EVNF0M703256N","ID_SERIAL_SHORT":"S4EVNF0M703256N","ID_WWN":"eui.0025385791b04175","MAJOR":"259","MINOR":"1","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"43271933"},"6":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/lvm-pv-uuid-tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0 /dev/disk/by-diskseq/9 /dev/disk/by-id/wwn-0x5002538c405dbf10","DEVNAME":"/dev/sda","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:0/end_device-8:0:0/target8:0:0/8:0:0:0/block/sda","DEVTYPE":"disk","DISKSEQ":"9","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_UUID_ENC":"tMMNAX-noqI-P0oS-9OEJ-7IR5-WoRL-N5K5Cv","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy0-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403550","ID_SERIAL_SHORT":"S2HRNX0J403550","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbf10","ID_WWN_WITH_EXTENSION":"0x5002538c405dbf10","MAJOR":"8","MINOR":"0","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45234812"},"7":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbce5 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0 /dev/disk/by-id/lvm-pv-uuid-oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu /dev/disk/by-diskseq/10 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","DEVNAME":"/dev/sdb","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:1/end_device-8:0:1/target8:0:1/8:0:1:0/block/sdb","DEVTYPE":"disk","DISKSEQ":"10","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_UUID_ENC":"oPUG7c-CMh3-oHQy-YRZP-8cNJ-uMIv-ceVPZu","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy1-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403335","ID_SERIAL_SHORT":"S2HRNX0J403335","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbce5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbce5","MAJOR":"8","MINOR":"16","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215406"},"8":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-id/wwn-0x5002538c405dbcd9 /dev/disk/by-diskseq/11 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333 /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0 /dev/disk/by-id/lvm-pv-uuid-tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","DEVNAME":"/dev/sdc","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:2/end_device-8:0:2/target8:0:2/8:0:2:0/block/sdc","DEVTYPE":"disk","DISKSEQ":"11","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_UUID_ENC":"tbguYd-sqom-3Okm-aJ0F-0F8N-2ALl-lo7ONW","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy2-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403333","ID_SERIAL_SHORT":"S2HRNX0J403333","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbcd9","ID_WWN_WITH_EXTENSION":"0x5002538c405dbcd9","MAJOR":"8","MINOR":"32","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45198824"},"9":{"CURRENT_TAGS":":systemd:","DEVLINKS":"/dev/disk/by-diskseq/12 /dev/disk/by-id/wwn-0x5002538c405dbdc5 /dev/disk/by-id/lvm-pv-uuid-Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz /dev/disk/by-path/pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0 /dev/disk/by-id/ata-SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","DEVNAME":"/dev/sdd","DEVPATH":"/devices/pci0000:00/0000:00:03.1/0000:05:00.0/host8/port-8:0/expander-8:0/port-8:0:3/end_device-8:0:3/target8:0:3/8:0:3:0/block/sdd","DEVTYPE":"disk","DISKSEQ":"12","ID_ATA":"1","ID_ATA_DOWNLOAD_MICROCODE":"1","ID_ATA_FEATURE_SET_HPA":"1","ID_ATA_FEATURE_SET_HPA_ENABLED":"1","ID_ATA_FEATURE_SET_PM":"1","ID_ATA_FEATURE_SET_PM_ENABLED":"1","ID_ATA_FEATURE_SET_SECURITY":"1","ID_ATA_FEATURE_SET_SECURITY_ENABLED":"0","ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN":"32","ID_ATA_FEATURE_SET_SMART":"1","ID_ATA_FEATURE_SET_SMART_ENABLED":"1","ID_ATA_ROTATION_RATE_RPM":"0","ID_ATA_SATA":"1","ID_ATA_WRITE_CACHE":"1","ID_ATA_WRITE_CACHE_ENABLED":"1","ID_BUS":"ata","ID_FS_TYPE":"LVM2_member","ID_FS_USAGE":"raid","ID_FS_UUID":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_UUID_ENC":"Lpxa0X-i8MT-EYWV-J7yQ-r5x7-S99u-jLf8bz","ID_FS_VERSION":"LVM2 001","ID_MODEL":"SAMSUNG_MZ7KM240HAGR-00005","ID_MODEL_ENC":"SAMSUNG\\x20MZ7KM240HAGR-00005\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20","ID_PATH":"pci-0000:05:00.0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_PATH_TAG":"pci-0000_05_00_0-sas-exp0x500304801f3f7f7f-phy5-lun-0","ID_REVISION":"GXM1103Q","ID_SERIAL":"SAMSUNG_MZ7KM240HAGR-00005_S2HRNX0J403419","ID_SERIAL_SHORT":"S2HRNX0J403419","ID_TYPE":"disk","ID_WWN":"0x5002538c405dbdc5","ID_WWN_WITH_EXTENSION":"0x5002538c405dbdc5","MAJOR":"8","MINOR":"48","SUBSYSTEM":"block","TAGS":":systemd:","USEC_INITIALIZED":"45215283"}},"nics":{"eno1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.0/net/eno1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN1","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno1","ID_NET_NAME_MAC":"enxb42e99acadb4","ID_NET_NAME_ONBOARD":"eno1","ID_NET_NAME_PATH":"enp194s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.0","ID_PATH_TAG":"pci-0000_c2_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"5","INTERFACE":"eno1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno1","TAGS":":systemd:","USEC_INITIALIZED":"45212091"},"eno2":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:03.5/0000:c2:00.1/net/eno2","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LABEL_ONBOARD":"Onboard LAN2","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"eno2","ID_NET_NAME_MAC":"enxb42e99acadb5","ID_NET_NAME_ONBOARD":"eno2","ID_NET_NAME_PATH":"enp194s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"GIGA-BYTE TECHNOLOGY CO.,LTD.","ID_PATH":"pci-0000:c2:00.1","ID_PATH_TAG":"pci-0000_c2_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"6","INTERFACE":"eno2","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/eno2","TAGS":":systemd:","USEC_INITIALIZED":"45128159"},"enp129s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.0/net/enp129s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f0np0","ID_NET_NAME_MAC":"enx1c34da5c5e24","ID_NET_NAME_PATH":"enp129s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.0","ID_PATH_TAG":"pci-0000_81_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"7","INTERFACE":"enp129s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47752091"},"enp129s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:80/0000:80:01.1/0000:81:00.1/net/enp129s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27710 Family [ConnectX-4 Lx] (MCX4421A-ACQN ConnectX-4 Lx EN OCP,2x25G)","ID_MODEL_ID":"0x1015","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp129s0f1np1","ID_NET_NAME_MAC":"enx1c34da5c5e25","ID_NET_NAME_PATH":"enp129s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:81:00.1","ID_PATH_TAG":"pci-0000_81_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"8","INTERFACE":"enp129s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp129s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47716100"},"enp193s0f0np0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.0/net/enp193s0f0np0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f0np0","ID_NET_NAME_MAC":"enx248a071e05bc","ID_NET_NAME_PATH":"enp193s0f0np0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.0","ID_PATH_TAG":"pci-0000_c1_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"9","INTERFACE":"enp193s0f0np0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f0np0","TAGS":":systemd:","USEC_INITIALIZED":"47784094"},"enp193s0f1np1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.1/net/enp193s0f1np1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"MT27700 Family [ConnectX-4]","ID_MODEL_ID":"0x1013","ID_NET_DRIVER":"mlx5_core","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp193s0f1np1","ID_NET_NAME_MAC":"enx248a071e05bd","ID_NET_NAME_PATH":"enp193s0f1np1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Mellanox Technologies, Inc.","ID_PATH":"pci-0000:c1:00.1","ID_PATH_TAG":"pci-0000_c1_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Mellanox Technologies","ID_VENDOR_ID":"0x15b3","IFINDEX":"10","INTERFACE":"enp193s0f1np1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp193s0f1np1","TAGS":":systemd:","USEC_INITIALIZED":"47820155"},"enp65s0f0":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.0/net/enp65s0f0","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f0","ID_NET_NAME_MAC":"enxa0369f0ab382","ID_NET_NAME_PATH":"enp65s0f0","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.0","ID_PATH_TAG":"pci-0000_41_00_0","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"3","INTERFACE":"enp65s0f0","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f0","TAGS":":systemd:","USEC_INITIALIZED":"45176103"},"enp65s0f1":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:03.1/0000:41:00.1/net/enp65s0f1","ID_BUS":"pci","ID_MODEL_FROM_DATABASE":"I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)","ID_MODEL_ID":"0x1521","ID_NET_DRIVER":"igb","ID_NET_LINK_FILE":"/usr/lib/systemd/network/99-default.link","ID_NET_NAME":"enp65s0f1","ID_NET_NAME_MAC":"enxa0369f0ab383","ID_NET_NAME_PATH":"enp65s0f1","ID_NET_NAMING_SCHEME":"v252","ID_OUI_FROM_DATABASE":"Intel Corporate","ID_PATH":"pci-0000:41:00.1","ID_PATH_TAG":"pci-0000_41_00_1","ID_PCI_CLASS_FROM_DATABASE":"Network controller","ID_PCI_SUBCLASS_FROM_DATABASE":"Ethernet controller","ID_VENDOR_FROM_DATABASE":"Intel Corporation","ID_VENDOR_ID":"0x8086","IFINDEX":"4","INTERFACE":"enp65s0f1","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enp65s0f1","TAGS":":systemd:","USEC_INITIALIZED":"45260218"},"enxaa0c304b6362":{"CURRENT_TAGS":":systemd:","DEVPATH":"/devices/pci0000:40/0000:40:08.1/0000:43:00.3/usb3/3-2/3-2.4/3-2.4.3/3-2.4.3:2.0/net/enxaa0c304b6362","ID_BUS":"usb","ID_MODEL":"Virtual_Ethernet","ID_MODEL_ENC":"Virtual\\x20Ethernet","ID_MODEL_ID":"ffb0","ID_NET_DRIVER":"cdc_ether","ID_NET_LINK_FILE":"/usr/lib/systemd/network/73-usb-net-by-mac.link","ID_NET_NAME":"enxaa0c304b6362","ID_NET_NAME_MAC":"enxaa0c304b6362","ID_NET_NAME_PATH":"enp67s0f3u2u4u3c2","ID_NET_NAMING_SCHEME":"v252","ID_PATH":"pci-0000:43:00.3-usb-0:2.4.3:2.0","ID_PATH_TAG":"pci-0000_43_00_3-usb-0_2_4_3_2_0","ID_REVISION":"0100","ID_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_SERIAL_SHORT":"1234567890","ID_TYPE":"generic","ID_USB_CLASS_FROM_DATABASE":"Communications","ID_USB_DRIVER":"cdc_ether","ID_USB_INTERFACES":":0202ff:0a0000:020600:","ID_USB_INTERFACE_NUM":"00","ID_USB_MODEL":"Virtual_Ethernet","ID_USB_MODEL_ENC":"Virtual\\x20Ethernet","ID_USB_MODEL_ID":"ffb0","ID_USB_REVISION":"0100","ID_USB_SERIAL":"American_Megatrends_Inc._Virtual_Ethernet_1234567890","ID_USB_SERIAL_SHORT":"1234567890","ID_USB_TYPE":"generic","ID_USB_VENDOR":"American_Megatrends_Inc.","ID_USB_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_USB_VENDOR_ID":"046b","ID_VENDOR":"American_Megatrends_Inc.","ID_VENDOR_ENC":"American\\x20Megatrends\\x20Inc.","ID_VENDOR_FROM_DATABASE":"American Megatrends, Inc.","ID_VENDOR_ID":"046b","IFINDEX":"2","INTERFACE":"enxaa0c304b6362","SUBSYSTEM":"net","SYSTEMD_ALIAS":"/sys/subsystem/net/devices/enxaa0c304b6362","TAGS":":systemd:","USEC_INITIALIZED":"44748106"}}}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 14/30] auto-installer: add auto-installer binary
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (12 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 13/30] auto-installer: add tests for answer file parsing Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 15/30] auto-installer: add fetch answer binary Aaron Lauterer
` (17 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
It expects the contents of an answer file via stdin. It will then be
parsed and the JSON for the low level installer is generated.
It then calls the low level installer directly.
The output of the installaton progress is kept rather simple for now.
If configured in the answer file, commands will be run pre and post the
low level installer.
It also logs everything to the logfile, currently
'/tmp/auto_installer.log'.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 9 +-
.../src/bin/proxmox-auto-installer.rs | 195 ++++++++++++++++++
2 files changed, 201 insertions(+), 3 deletions(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
diff --git a/Makefile b/Makefile
index 4f140ca..3ac5769 100644
--- a/Makefile
+++ b/Makefile
@@ -18,7 +18,9 @@ INSTALLER_SOURCES=$(shell git ls-files) country.dat
PREFIX = /usr
BINDIR = $(PREFIX)/bin
-USR_BIN := proxmox-tui-installer
+USR_BIN := \
+ proxmox-tui-installer\
+ proxmox-auto-installer
COMPILED_BINS := \
$(addprefix $(CARGO_COMPILEDIR)/,$(USR_BIN))
@@ -99,7 +101,7 @@ VARLIBDIR=$(DESTDIR)/var/lib/proxmox-installer
HTMLDIR=$(VARLIBDIR)/html/common
.PHONY: install
-install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer
+install: $(INSTALLER_SOURCES) $(COMPILED_BINS)
$(MAKE) -C banner install
$(MAKE) -C Proxmox install
install -D -m 644 interfaces $(DESTDIR)/etc/network/interfaces
@@ -118,7 +120,8 @@ install: $(INSTALLER_SOURCES) $(CARGO_COMPILEDIR)/proxmox-tui-installer
$(COMPILED_BINS): cargo-build
.PHONY: cargo-build
cargo-build:
- $(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer $(CARGO_BUILD_ARGS)
+ $(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
+ --package proxmox-auto-installer --bin proxmox-auto-installer $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
new file mode 100644
index 0000000..f43b12f
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
@@ -0,0 +1,195 @@
+use anyhow::{anyhow, bail, Error, Result};
+use log::{error, info, LevelFilter};
+use std::{
+ env,
+ io::{BufRead, BufReader, Write},
+ path::PathBuf,
+ process::ExitCode,
+};
+
+use proxmox_installer_common::setup::{
+ installer_setup, read_json, spawn_low_level_installer, LocaleInfo, RuntimeInfo, SetupInfo,
+};
+
+use proxmox_auto_installer::{
+ answer::Answer,
+ log::AutoInstLogger,
+ udevinfo::UdevInfo,
+ utils,
+ utils::{parse_answer, LowLevelMessage},
+};
+
+static LOGGER: AutoInstLogger = AutoInstLogger;
+
+pub fn init_log() -> Result<()> {
+ AutoInstLogger::init("/tmp/auto_installer.log")?;
+ log::set_logger(&LOGGER)
+ .map(|()| log::set_max_level(LevelFilter::Info))
+ .map_err(|err| anyhow!(err))
+}
+
+fn auto_installer_setup(in_test_mode: bool) -> Result<(Answer, UdevInfo)> {
+ let base_path = if in_test_mode { "./testdir" } else { "/" };
+ let mut path = PathBuf::from(base_path);
+
+ path.push("run");
+ path.push("proxmox-installer");
+
+ let udev_info: UdevInfo = {
+ let mut path = path.clone();
+ path.push("run-env-udev.json");
+
+ read_json(&path).map_err(|err| anyhow!("Failed to retrieve udev info details: {err}"))?
+ };
+
+ let mut buffer = String::new();
+ let lines = std::io::stdin().lock().lines();
+ for line in lines {
+ buffer.push_str(&line.unwrap());
+ buffer.push('\n');
+ }
+
+ let answer: Answer =
+ toml::from_str(&buffer).map_err(|err| anyhow!("Failed parsing answer file: {err}"))?;
+
+ Ok((answer, udev_info))
+}
+
+fn main() -> ExitCode {
+ if let Err(err) = init_log() {
+ panic!("could not initilize logging: {}", err);
+ }
+
+ let in_test_mode = match env::args().nth(1).as_deref() {
+ Some("-t") => true,
+ // Always force the test directory in debug builds
+ _ => cfg!(debug_assertions),
+ };
+ info!("Starting auto installer");
+
+ let (setup_info, locales, runtime_info) = match installer_setup(in_test_mode) {
+ Ok(result) => result,
+ Err(err) => {
+ error!("Installer setup error: {err}");
+ return ExitCode::FAILURE;
+ }
+ };
+
+ let (answer, udevadm_info) = match auto_installer_setup(in_test_mode) {
+ Ok(result) => result,
+ Err(err) => {
+ error!("Autoinstaller setup error: {err}");
+ return ExitCode::FAILURE;
+ }
+ };
+
+ match utils::run_cmds("Pre", &answer.global.pre_commands) {
+ Ok(_) => (),
+ Err(err) => {
+ error!("Error when running Pre-Commands: {}", err);
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ };
+ match run_installation(&answer, &locales, &runtime_info, &udevadm_info, &setup_info) {
+ Ok(_) => info!("Installation done."),
+ Err(err) => {
+ error!("Installation failed: {err}");
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ }
+ match utils::run_cmds("Post", &answer.global.post_commands) {
+ Ok(_) => (),
+ Err(err) => {
+ error!("Error when running Post-Commands: {}", err);
+ return exit_failure(answer.global.reboot_on_error);
+ }
+ };
+ ExitCode::SUCCESS
+}
+
+/// When we exit with a failure, the installer will not automatically reboot.
+/// Default value for reboot_on_error is false
+fn exit_failure(reboot_on_error: bool) -> ExitCode {
+ if reboot_on_error {
+ ExitCode::SUCCESS
+ } else {
+ ExitCode::FAILURE
+ }
+}
+
+fn run_installation(
+ answer: &Answer,
+ locales: &LocaleInfo,
+ runtime_info: &RuntimeInfo,
+ udevadm_info: &UdevInfo,
+ setup_info: &SetupInfo,
+) -> Result<()> {
+ let config = parse_answer(answer, udevadm_info, runtime_info, locales, setup_info)?;
+ info!("Calling low-level installer");
+
+ let mut child = match spawn_low_level_installer(false) {
+ Ok(child) => child,
+ Err(err) => {
+ bail!("Low level installer could not be started: {}", err);
+ }
+ };
+
+ let mut cur_counter = 111;
+ let mut inner = || -> Result<()> {
+ let reader = child
+ .stdout
+ .take()
+ .map(BufReader::new)
+ .ok_or(anyhow!("failed to get stdout reader"))?;
+ let mut writer = child
+ .stdin
+ .take()
+ .ok_or(anyhow!("failed to get stdin writer"))?;
+
+ serde_json::to_writer(&mut writer, &config)
+ .map_err(|err| anyhow!("failed to serialize install config: {err}"))?;
+ writeln!(writer).map_err(|err| anyhow!("failed to write install config: {err}"))?;
+
+ for line in reader.lines() {
+ let line = match line {
+ Ok(line) => line,
+ Err(_) => break,
+ };
+ let msg = match serde_json::from_str::<LowLevelMessage>(&line) {
+ Ok(msg) => msg,
+ Err(_) => {
+ // Not a fatal error, so don't abort the installation by returning
+ continue;
+ }
+ };
+
+ match msg.clone() {
+ LowLevelMessage::Info { message } => info!("{message}"),
+ LowLevelMessage::Error { message } => error!("{message}"),
+ LowLevelMessage::Prompt { query } => {
+ bail!("Got interactive prompt I cannot answer: {query}")
+ }
+ LowLevelMessage::Progress { ratio, text: _ } => {
+ let counter = (ratio * 100.).floor() as usize;
+ if counter != cur_counter {
+ cur_counter = counter;
+ info!("Progress: {counter:>3}%");
+ }
+ }
+ LowLevelMessage::Finished { state, message } => {
+ if state == "err" {
+ bail!("{message}");
+ }
+ info!("Finished: '{state}' {message}");
+ }
+ };
+ }
+ Ok(())
+ };
+ match inner() {
+ Err(err) => Err(Error::msg(format!(
+ "low level installer returned early: {err}"
+ ))),
+ _ => Ok(()),
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 15/30] auto-installer: add fetch answer binary
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (13 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 14/30] auto-installer: add auto-installer binary Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 16/30] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
` (16 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
it is supposed to be run first and fetch an answer file.
The initial implementation searches for a partition/filesystem called
'proxmoxinst' or 'PROXMOXINST' with an 'answer.toml' file in the root
directory.
Once it has an answer file, it will call the 'proxmox-auto-installer'
and pipe in the contents via stdin.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 4 +-
.../src/bin/proxmox-fetch-answer.rs | 71 ++++++++++++++++
.../src/fetch_plugins/mod.rs | 2 +
.../src/fetch_plugins/partition.rs | 32 ++++++++
.../src/fetch_plugins/utils.rs | 81 +++++++++++++++++++
proxmox-auto-installer/src/lib.rs | 1 +
6 files changed, 190 insertions(+), 1 deletion(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils.rs
diff --git a/Makefile b/Makefile
index 3ac5769..e44450e 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,7 @@ PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
proxmox-tui-installer\
+ proxmox-fetch-answer\
proxmox-auto-installer
COMPILED_BINS := \
@@ -121,7 +122,8 @@ $(COMPILED_BINS): cargo-build
.PHONY: cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
- --package proxmox-auto-installer --bin proxmox-auto-installer $(CARGO_BUILD_ARGS)
+ --package proxmox-auto-installer --bin proxmox-auto-installer \
+ --bin proxmox-fetch-answer $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
new file mode 100644
index 0000000..a3681a2
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
@@ -0,0 +1,71 @@
+use anyhow::{anyhow, Error, Result};
+use log::{error, info, LevelFilter};
+use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger};
+use std::io::Write;
+use std::process::{Command, ExitCode, Stdio};
+
+static LOGGER: AutoInstLogger = AutoInstLogger;
+
+pub fn init_log() -> Result<()> {
+ AutoInstLogger::init("/tmp/fetch_answer.log")?;
+ log::set_logger(&LOGGER)
+ .map(|()| log::set_max_level(LevelFilter::Info))
+ .map_err(|err| anyhow!(err))
+}
+
+fn fetch_answer() -> Result<String> {
+ match FetchFromPartition::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file from partition failed: {err}"),
+ }
+ // TODO: add more options to get an answer file, e.g. download from url where url could be
+ // fetched via txt records on predefined subdomain, kernel param, dhcp option, ...
+
+ Err(Error::msg("Could not find any answer file!"))
+}
+
+fn main() -> ExitCode {
+ if let Err(err) = init_log() {
+ panic!("could not initialize logging: {err}");
+ }
+
+ info!("Fetching answer file");
+ let answer = match fetch_answer() {
+ Ok(answer) => answer,
+ Err(err) => {
+ error!("Aborting: {}", err);
+ return ExitCode::FAILURE;
+ }
+ };
+
+ let mut child = match Command::new("proxmox-auto-installer")
+ .stdout(Stdio::inherit())
+ .stdin(Stdio::piped())
+ .stderr(Stdio::null())
+ .spawn()
+ {
+ Ok(child) => child,
+ Err(err) => panic!("Failed to start automatic installation: {err}"),
+ };
+
+ let mut stdin = child.stdin.take().expect("Failed to open stdin");
+ std::thread::spawn(move || {
+ stdin
+ .write_all(answer.as_bytes())
+ .expect("Failed to write to stdin");
+ });
+
+ match child.wait() {
+ Ok(status) => {
+ if status.success() {
+ ExitCode::SUCCESS
+ } else {
+ ExitCode::FAILURE // Will be trapped
+ }
+ }
+ Err(err) => {
+ error!("Auto installer exited: {err}");
+ ExitCode::FAILURE
+ }
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
new file mode 100644
index 0000000..11d6937
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -0,0 +1,2 @@
+pub mod partition;
+mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/partition.rs b/proxmox-auto-installer/src/fetch_plugins/partition.rs
new file mode 100644
index 0000000..0c47a62
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/partition.rs
@@ -0,0 +1,32 @@
+use anyhow::{Error, Result};
+use std::{fs::read_to_string, path::Path};
+use log::info;
+
+use crate::fetch_plugins::utils::mount_proxmoxinst_part;
+
+static ANSWER_FILE: &str = "answer.toml";
+
+pub struct FetchFromPartition;
+
+impl FetchFromPartition {
+ /// Returns the contents of the answer file
+ pub fn get_answer() -> Result<String> {
+ info!("Checking for answer file on partition.");
+ let mount_path = mount_proxmoxinst_part()?;
+ let answer = Self::get_answer_file(&mount_path)?;
+ info!("Found answer file on partition.");
+ Ok(answer)
+ }
+
+ /// Searches for answer file and returns contents if found
+ fn get_answer_file(mount_path: &str) -> Result<String> {
+ let answer_path = Path::new(mount_path).join(ANSWER_FILE);
+ match answer_path.try_exists() {
+ Ok(true) => Ok(read_to_string(answer_path)?),
+ _ => Err(Error::msg(format!(
+ "could not find answer file expected at: {}",
+ answer_path.display()
+ ))),
+ }
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils.rs
new file mode 100644
index 0000000..f2a8e74
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils.rs
@@ -0,0 +1,81 @@
+use anyhow::{bail, Error, Result};
+use log::{info, warn};
+use std::{
+ fs::{self, create_dir_all},
+ path::{Path, PathBuf},
+ process::Command,
+};
+
+static ANSWER_MP: &str = "/mnt/answer";
+static PARTLABEL: &str = "proxmoxinst";
+static SEARCH_PATH: &str = "/dev/disk/by-label";
+
+/// Searches for upper and lower case existence of the partlabel in the search_path
+///
+/// # Arguemnts
+/// * `partlabel_source` - Partition Label, used as upper and lower case
+/// * `search_path` - Path where to search for the partiiton label
+pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<PathBuf> {
+ let partlabel = partlabel_source.to_uppercase();
+ let path = Path::new(search_path).join(&partlabel);
+ match path.try_exists() {
+ Ok(true) => {
+ info!("Found partition with label '{partlabel}'");
+ return Ok(path);
+ }
+ Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
+ }
+
+ let partlabel = partlabel_source.to_lowercase();
+ let path = Path::new(search_path).join(&partlabel);
+ match path.try_exists() {
+ Ok(true) => {
+ info!("Found partition with label '{partlabel}'");
+ return Ok(path);
+ }
+ Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
+ }
+ bail!("Could not detect upper or lower case labels for '{partlabel_source}'");
+}
+
+/// Will search and mount a partition/FS labeled proxmoxinst in lower or uppercase to ANSWER_MP;
+pub fn mount_proxmoxinst_part() -> Result<String> {
+ if let Ok(true) = check_if_mounted(ANSWER_MP) {
+ info!("Skipping: '{ANSWER_MP}' is already mounted.");
+ return Ok(ANSWER_MP.into());
+ }
+ let part_path = scan_partlabels(PARTLABEL, SEARCH_PATH)?;
+ info!("Mounting partition at {ANSWER_MP}");
+ // create dir for mountpoint
+ create_dir_all(ANSWER_MP)?;
+ match Command::new("mount")
+ .args(["-o", "ro"])
+ .arg(part_path)
+ .arg(ANSWER_MP)
+ .output()
+ {
+ Ok(output) => {
+ if output.status.success() {
+ Ok(ANSWER_MP.into())
+ } else {
+ warn!("Error mounting: {}", String::from_utf8(output.stderr)?);
+ Ok(ANSWER_MP.into())
+ }
+ }
+ Err(err) => Err(Error::msg(format!("Error mounting: {err}"))),
+ }
+}
+
+fn check_if_mounted(target_path: &str) -> Result<bool> {
+ let mounts = fs::read_to_string("/proc/mounts")?;
+ for line in mounts.lines() {
+ if let Some(mp) = line.split(' ').nth(1) {
+ if mp == target_path {
+ return Ok(true);
+ }
+ }
+ }
+ Ok(false)
+}
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 6636cc7..0a153b2 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,4 +1,5 @@
pub mod answer;
+pub mod fetch_plugins;
pub mod log;
pub mod udevinfo;
pub mod utils;
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 16/30] unconfigured: add proxauto as option to start auto installer
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (14 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 15/30] auto-installer: add fetch answer binary Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 17/30] auto-installer: use glob crate for pattern matching Aaron Lauterer
` (15 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
unconfigured.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/unconfigured.sh b/unconfigured.sh
index 2b371f0..f02336a 100755
--- a/unconfigured.sh
+++ b/unconfigured.sh
@@ -5,6 +5,7 @@ trap "err_reboot" ERR
# NOTE: we nowadays get exec'd by the initrd's PID 1, so we're the new PID 1
parse_cmdline() {
+ proxauto=0
proxdebug=0
proxtui=0
serial=0
@@ -17,6 +18,9 @@ parse_cmdline() {
proxtui)
proxtui=1
;;
+ proxauto)
+ proxauto=1
+ ;;
console=ttyS*)
serial=1
;;
@@ -224,6 +228,10 @@ setsid /sbin/agetty -a root --noclear tty3 &
if [ $proxtui -ne 0 ]; then
echo "Starting the TUI installer"
/usr/bin/proxmox-tui-installer 2>/dev/tty2
+elif [ $proxauto -ne 0 ]; then
+ /usr/bin/proxmox-low-level-installer dump-udev
+ echo "Starting automatic installation"
+ /usr/bin/proxmox-fetch-answer
else
echo "Starting the installer GUI - see tty2 (CTRL+ALT+F2) for any errors..."
xinit -- -dpi "$DPI" -s 0 >/dev/tty2 2>&1
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 17/30] auto-installer: use glob crate for pattern matching
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (15 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 16/30] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 18/30] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
` (14 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 1 +
proxmox-auto-installer/src/utils.rs | 46 +++++++++++------------------
2 files changed, 18 insertions(+), 29 deletions(-)
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index b1d3e7a..741794a 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -9,6 +9,7 @@ homepage = "https://www.proxmox.com"
[dependencies]
anyhow = "1.0"
+glob = "0.3"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index ae28b1e..6101d66 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -1,4 +1,5 @@
use anyhow::{bail, Result};
+use glob::Pattern;
use log::info;
use std::{
collections::BTreeMap,
@@ -15,29 +16,11 @@ use proxmox_installer_common::{
};
use serde::Deserialize;
-/// Supports the globbing character '*' at the beginning, end or both of the pattern.
-/// Globbing within the pattern is not supported
-fn find_with_glob(pattern: &str, value: &str) -> bool {
- let globbing_symbol = '*';
- let mut start_glob = false;
- let mut end_glob = false;
- let mut pattern = pattern;
-
- if pattern.starts_with(globbing_symbol) {
- start_glob = true;
- pattern = &pattern[1..];
- }
-
- if pattern.ends_with(globbing_symbol) {
- end_glob = true;
- pattern = &pattern[..pattern.len() - 1]
- }
-
- match (start_glob, end_glob) {
- (true, true) => value.contains(pattern),
- (true, false) => value.ends_with(pattern),
- (false, true) => value.starts_with(pattern),
- _ => value == pattern,
+fn find_with_glob(pattern: &str, value: &str) -> Result<bool> {
+ let p = Pattern::new(pattern)?;
+ match p.matches(value) {
+ true => Ok(true),
+ false => Ok(false),
}
}
@@ -75,7 +58,7 @@ fn get_single_udev_index(
'outer: for (dev, dev_values) in udev_list {
for (filter_key, filter_value) in &filter {
for (udev_key, udev_value) in dev_values {
- if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value)? {
dev_index = Some(dev.clone());
break 'outer; // take first match
}
@@ -100,7 +83,7 @@ fn get_matched_udev_indexes(
let mut did_match_all = true;
for (filter_key, filter_value) in &filter {
for (udev_key, udev_value) in dev_values {
- if udev_key == filter_key && find_with_glob(filter_value, udev_value) {
+ if udev_key == filter_key && find_with_glob(filter_value, udev_value)? {
did_match_once = true;
} else if udev_key == filter_key {
did_match_all = false;
@@ -416,9 +399,14 @@ mod tests {
#[test]
fn test_glob_patterns() {
let test_value = "foobar";
- assert_eq!(find_with_glob("*bar", test_value), true);
- assert_eq!(find_with_glob("foo*", test_value), true);
- assert_eq!(find_with_glob("foobar", test_value), true);
- assert_eq!(find_with_glob("oobar", test_value), false);
+ assert_eq!(find_with_glob("*bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("foo*", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("foobar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("oobar", test_value).unwrap(), false);
+ assert_eq!(find_with_glob("f*bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f?bar", test_value).unwrap(), false);
+ assert_eq!(find_with_glob("fo?bar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f[!a]obar", test_value).unwrap(), true);
+ assert_eq!(find_with_glob("f[oa]obar", test_value).unwrap(), true);
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 18/30] auto-installer: utils: make get_udev_index functions public
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (16 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 17/30] auto-installer: use glob crate for pattern matching Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 19/30] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
` (13 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
because we will need to access them directly in the future from a
separate binary
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/src/utils.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index 6101d66..ea645ad 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -47,7 +47,7 @@ pub fn get_network_settings(
Ok(network_options)
}
-fn get_single_udev_index(
+pub fn get_single_udev_index(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
) -> Result<String> {
@@ -72,7 +72,7 @@ fn get_single_udev_index(
Ok(dev_index.unwrap())
}
-fn get_matched_udev_indexes(
+pub fn get_matched_udev_indexes(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
match_all: bool,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 19/30] auto-installer: add proxmox-autoinst-helper tool
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (17 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 18/30] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 20/30] common: add Display trait to ProxmoxProduct Aaron Lauterer
` (12 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
It can parse an answer file to check against syntax errors, test match
filters against the current hardware and list properties of the current
hardware to match against.
Since this tool should be able to run outside of the installer
environment, it does not rely on the device information provided by the
low-level installer. It instead fetches the list of disks and NICs by
itself.
The rules when a device is ignored, should match how the low-level
installer handles it.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Makefile | 3 +-
proxmox-auto-installer/Cargo.toml | 2 +
proxmox-auto-installer/src/answer.rs | 3 +-
.../src/bin/proxmox-autoinst-helper.rs | 340 ++++++++++++++++++
4 files changed, 346 insertions(+), 2 deletions(-)
create mode 100644 proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
diff --git a/Makefile b/Makefile
index e44450e..197a351 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,7 @@ PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
proxmox-tui-installer\
+ proxmox-autoinst-helper\
proxmox-fetch-answer\
proxmox-auto-installer
@@ -123,7 +124,7 @@ $(COMPILED_BINS): cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
- --bin proxmox-fetch-answer $(CARGO_BUILD_ARGS)
+ --bin proxmox-fetch-answer --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index 741794a..bb0b49c 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -9,6 +9,7 @@ homepage = "https://www.proxmox.com"
[dependencies]
anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
glob = "0.3"
proxmox-installer-common = { path = "../proxmox-installer-common" }
serde = { version = "1.0", features = ["derive"] }
@@ -16,3 +17,4 @@ serde_json = "1.0"
toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
+regex = "1.7"
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
index 0add95e..94cebb3 100644
--- a/proxmox-auto-installer/src/answer.rs
+++ b/proxmox-auto-installer/src/answer.rs
@@ -1,3 +1,4 @@
+use clap::ValueEnum;
use proxmox_installer_common::{
options::{BtrfsRaidLevel, FsType, ZfsChecksumOption, ZfsCompressOption, ZfsRaidLevel},
utils::{CidrAddress, Fqdn},
@@ -205,7 +206,7 @@ pub enum DiskSelection {
Selection(Vec<String>),
Filter(BTreeMap<String, String>),
}
-#[derive(Clone, Deserialize, Debug, PartialEq)]
+#[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)]
#[serde(rename_all = "lowercase")]
pub enum FilterMatch {
Any,
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
new file mode 100644
index 0000000..058d5ff
--- /dev/null
+++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
@@ -0,0 +1,340 @@
+use anyhow::{bail, Result};
+use clap::{Args, Parser, Subcommand, ValueEnum};
+use glob::Pattern;
+use regex::Regex;
+use serde::{Deserialize, Serialize};
+use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
+
+use proxmox_auto_installer::{
+ answer::Answer,
+ answer::FilterMatch,
+ utils::{get_matched_udev_indexes, get_single_udev_index},
+};
+
+/// This tool validates the format of an answer file. Additionally it can test match filters and
+/// print information on the properties to match against for the current hardware.
+#[derive(Parser, Debug)]
+#[command(author, version, about, long_about = None)]
+struct Cli {
+ #[command(subcommand)]
+ command: Commands,
+}
+
+#[derive(Subcommand, Debug)]
+enum Commands {
+ ValidateAnswer(CommandValidateAnswer),
+ Match(CommandMatch),
+ Info(CommandInfo),
+}
+
+/// Show device information that can be used for filters
+#[derive(Args, Debug)]
+struct CommandInfo {
+ /// For which device type information should be shown
+ #[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)]
+ device: AllDeviceTypes,
+}
+
+/// Test which devices the given filter matches against
+///
+/// Filters support the following syntax:
+/// ? Match a single character
+/// * Match any number of characters
+/// [a], [0-9] Specifc character or range of characters
+/// [!a] Negate a specific character of range
+///
+/// To avoid globbing characters being interpreted by the shell, use single quotes.
+/// Multiple filters can be defined.
+///
+/// Examples:
+/// Match disks against the serial number and device name, both must match:
+///
+/// proxmox-autoinst-helper match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*'
+#[derive(Args, Debug)]
+#[command(verbatim_doc_comment)]
+struct CommandMatch {
+ /// Device type to match the filter against
+ r#type: Devicetype,
+
+ /// Filter in the format KEY=VALUE where the key is the UDEV key and VALUE the filter string.
+ /// Multiple filters are possible, separated by a space.
+ filter: Vec<String>,
+
+ /// Defines if any filter or all filters must match.
+ #[arg(long, value_enum, default_value_t=FilterMatch::Any)]
+ filter_match: FilterMatch,
+}
+
+/// Validate if an answer file is formatted correctly.
+#[derive(Args, Debug)]
+struct CommandValidateAnswer {
+ /// Path to the answer file
+ path: PathBuf,
+ #[arg(short, long, default_value_t = false)]
+ debug: bool,
+}
+
+#[derive(Args, Debug)]
+struct GlobalOpts {
+ /// Output format
+ #[arg(long, short, value_enum)]
+ format: OutputFormat,
+}
+
+#[derive(Clone, Debug, ValueEnum, PartialEq)]
+enum AllDeviceTypes {
+ All,
+ Network,
+ Disk,
+}
+
+#[derive(Clone, Debug, ValueEnum)]
+enum Devicetype {
+ Network,
+ Disk,
+}
+
+#[derive(Clone, Debug, ValueEnum)]
+enum OutputFormat {
+ Pretty,
+ Json,
+}
+
+#[derive(Serialize)]
+struct Devs {
+ disks: Option<BTreeMap<String, BTreeMap<String, String>>>,
+ nics: Option<BTreeMap<String, BTreeMap<String, String>>>,
+}
+
+fn main() {
+ let args = Cli::parse();
+ let res = match &args.command {
+ Commands::Info(args) => info(args),
+ Commands::Match(args) => match_filter(args),
+ Commands::ValidateAnswer(args) => validate_answer(args),
+ };
+ if let Err(err) = res {
+ eprintln!("{err}");
+ std::process::exit(1);
+ }
+}
+
+fn info(args: &CommandInfo) -> Result<()> {
+ let mut devs = Devs {
+ disks: None,
+ nics: None,
+ };
+
+ if args.device == AllDeviceTypes::Network || args.device == AllDeviceTypes::All {
+ match get_nics() {
+ Ok(res) => devs.nics = Some(res),
+ Err(err) => bail!("Error getting NIC data: {err}"),
+ }
+ }
+ if args.device == AllDeviceTypes::Disk || args.device == AllDeviceTypes::All {
+ match get_disks() {
+ Ok(res) => devs.disks = Some(res),
+ Err(err) => bail!("Error getting disk data: {err}"),
+ }
+ }
+ println!("{}", serde_json::to_string_pretty(&devs).unwrap());
+ Ok(())
+}
+
+fn match_filter(args: &CommandMatch) -> Result<()> {
+ let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type {
+ Devicetype::Disk => get_disks().unwrap(),
+ Devicetype::Network => get_nics().unwrap(),
+ };
+ // parse filters
+
+ let mut filters: BTreeMap<String, String> = BTreeMap::new();
+
+ for f in &args.filter {
+ match f.split_once('=') {
+ Some((key, value)) => {
+ if key.is_empty() || value.is_empty() {
+ bail!("Filter key or value is empty in filter: '{f}'");
+ }
+ filters.insert(String::from(key), String::from(value));
+ }
+ None => {
+ bail!("Could not find separator '=' in filter: '{f}'");
+ }
+ }
+ }
+
+ // align return values
+ let result = match args.r#type {
+ Devicetype::Disk => {
+ get_matched_udev_indexes(filters, &devs, args.filter_match == FilterMatch::All)
+ }
+ Devicetype::Network => get_single_udev_index(filters, &devs).map(|r| vec![r]),
+ };
+
+ match result {
+ Ok(result) => println!("{}", serde_json::to_string_pretty(&result).unwrap()),
+ Err(err) => bail!("Error matching filters: {err}"),
+ }
+ Ok(())
+}
+
+fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
+ let mut file = match fs::File::open(&args.path) {
+ Ok(file) => file,
+ Err(err) => bail!(
+ "Opening answer file '{}' failed: {err}",
+ args.path.display()
+ ),
+ };
+ let mut contents = String::new();
+ if let Err(err) = file.read_to_string(&mut contents) {
+ bail!("Reading from file '{}' failed: {err}", args.path.display());
+ }
+
+ let answer: Answer = match toml::from_str(&contents) {
+ Ok(answer) => {
+ println!("The file was parsed successfully, no syntax errors found!");
+ answer
+ }
+ Err(err) => bail!("Error parsing answer file: {err}"),
+ };
+ if args.debug {
+ println!("Parsed data from answer file:\n{:#?}", answer);
+ }
+ Ok(())
+}
+
+fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
+ let unwantend_block_devs = vec![
+ "ram[0-9]*",
+ "loop[0-9]*",
+ "md[0-9]*",
+ "dm-*",
+ "fd[0-9]*",
+ "sr[0-9]*",
+ ];
+
+ // compile Regex here once and not inside the loop
+ let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?;
+ let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?;
+ let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?;
+
+ let re_name = Regex::new(r"(?m)^N: (.*)$")?;
+ let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
+
+ let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
+
+ 'outer: for entry in fs::read_dir("/sys/block")? {
+ let entry = entry.unwrap();
+ let filename = entry.file_name().into_string().unwrap();
+
+ for p in &unwantend_block_devs {
+ if Pattern::new(p)?.matches(&filename) {
+ continue 'outer;
+ }
+ }
+
+ let output = match get_udev_properties(&entry.path()) {
+ Ok(output) => output,
+ Err(err) => {
+ eprint!("{err}");
+ continue 'outer;
+ }
+ };
+
+ if !re_disk.is_match(&output) {
+ continue 'outer;
+ };
+ if re_cdrom.is_match(&output) {
+ continue 'outer;
+ };
+ if re_iso9660.is_match(&output) {
+ continue 'outer;
+ };
+
+ let mut name = filename;
+ if let Some(cap) = re_name.captures(&output) {
+ if let Some(res) = cap.get(1) {
+ name = String::from(res.as_str());
+ }
+ }
+
+ let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+
+ for line in output.lines() {
+ if let Some(caps) = re_props.captures(line) {
+ let key = String::from(caps.get(1).unwrap().as_str());
+ let value = String::from(caps.get(2).unwrap().as_str());
+ udev_props.insert(key, value);
+ }
+ }
+
+ disks.insert(name, udev_props);
+ }
+ Ok(disks)
+}
+
+#[derive(Deserialize, Debug)]
+struct IpLinksInfo {
+ ifname: String,
+}
+
+fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
+ let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
+ let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
+
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ for link in links {
+ let path = format!("/sys/class/net/{link}");
+
+ let output = match get_udev_properties(&PathBuf::from(path)) {
+ Ok(output) => output,
+ Err(err) => {
+ eprint!("{err}");
+ continue;
+ }
+ };
+
+ let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+
+ for line in output.lines() {
+ if let Some(caps) = re_props.captures(line) {
+ let key = String::from(caps.get(1).unwrap().as_str());
+ let value = String::from(caps.get(2).unwrap().as_str());
+ udev_props.insert(key, value);
+ }
+ }
+
+ nics.insert(link, udev_props);
+ }
+ Ok(nics)
+}
+
+fn get_udev_properties(path: &PathBuf) -> Result<String> {
+ let udev_output = Command::new("udevadm")
+ .arg("info")
+ .arg("--path")
+ .arg(path)
+ .arg("--query")
+ .arg("all")
+ .output()?;
+ if !udev_output.status.success() {
+ bail!("could not run udevadm successfully for {}", path.display());
+ }
+ Ok(String::from_utf8(udev_output.stdout)?)
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 20/30] common: add Display trait to ProxmoxProduct
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (18 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 19/30] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 21/30] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
` (11 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 8432a2c..25d0e9e 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -38,6 +38,16 @@ impl ProxmoxProduct {
}
}
+impl fmt::Display for ProxmoxProduct {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ match self {
+ Self::PVE => write!(f, "pve"),
+ Self::PMG => write!(f, "pmg"),
+ Self::PBS => write!(f, "pbs"),
+ }
+ }
+}
+
#[derive(Clone, Deserialize)]
pub struct ProductConfig {
pub fullname: String,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 21/30] auto-installer: fetch: add gathering of system identifiers and restructure code
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (19 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 20/30] common: add Display trait to ProxmoxProduct Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 22/30] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
` (10 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
They will be used as payload when POSTing a request for an answer file. The
idea is, that with this information, it should be possible to identify
the system and generate a matching answer file on the fly.
Many of these properties can also be found on the machine or packaging
of the machine and could therefore be scanned into a database.
Identifiers are the following properties from `dmidecode` sections 1, 2,
and 3:
* Asset Tag
* Product Name
* Serial Number
* SKU Number
* UUID
As well as a list of the MAC addresses of all the NICs and the product
type: pve, pmg, pbs.
Since we now have more than a simple utils.rs module in the fetch
plugins, it, and the additional fetch plugin utilities are placed in
their own directory.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/fetch_plugins/mod.rs | 2 +-
.../fetch_plugins/{utils.rs => utils/mod.rs} | 43 ++++++++--
.../src/fetch_plugins/utils/sysinfo.rs | 81 +++++++++++++++++++
3 files changed, 119 insertions(+), 7 deletions(-)
rename proxmox-auto-installer/src/fetch_plugins/{utils.rs => utils/mod.rs} (72%)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
index 11d6937..6f1e8a2 100644
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -1,2 +1,2 @@
pub mod partition;
-mod utils;
+pub mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
similarity index 72%
rename from proxmox-auto-installer/src/fetch_plugins/utils.rs
rename to proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
index f2a8e74..b3e9dad 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
@@ -1,5 +1,7 @@
-use anyhow::{bail, Error, Result};
+use anyhow::{Error, Result};
use log::{info, warn};
+use serde::Deserialize;
+use serde_json;
use std::{
fs::{self, create_dir_all},
path::{Path, PathBuf},
@@ -10,6 +12,8 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
+pub mod sysinfo;
+
/// Searches for upper and lower case existence of the partlabel in the search_path
///
/// # Arguemnts
@@ -20,10 +24,10 @@ pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<Path
let path = Path::new(search_path).join(&partlabel);
match path.try_exists() {
Ok(true) => {
- info!("Found partition with label '{partlabel}'");
+ info!("Found partition with label '{}'", partlabel);
return Ok(path);
}
- Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Ok(false) => info!("Did not detect partition with label '{}'", partlabel),
Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
}
@@ -31,13 +35,15 @@ pub fn scan_partlabels(partlabel_source: &str, search_path: &str) -> Result<Path
let path = Path::new(search_path).join(&partlabel);
match path.try_exists() {
Ok(true) => {
- info!("Found partition with label '{partlabel}'");
+ info!("Found partition with label '{}'", partlabel);
return Ok(path);
}
- Ok(false) => info!("Did not detect partition with label '{partlabel}'"),
+ Ok(false) => info!("Did not detect partition with label '{}'", partlabel),
Err(err) => info!("Encountered issue, accessing '{}': {}", path.display(), err),
}
- bail!("Could not detect upper or lower case labels for '{partlabel_source}'");
+ Err(Error::msg(format!(
+ "Could not detect upper or lower case labels for '{partlabel_source}'"
+ )))
}
/// Will search and mount a partition/FS labeled proxmoxinst in lower or uppercase to ANSWER_MP;
@@ -79,3 +85,28 @@ fn check_if_mounted(target_path: &str) -> Result<bool> {
}
Ok(false)
}
+
+#[derive(Deserialize, Debug)]
+struct IpLinksUdevInfo {
+ ifname: String,
+}
+
+/// Returns vec of usable NICs
+pub fn get_nic_list() -> Result<Vec<String>> {
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksUdevInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ Ok(links)
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
new file mode 100644
index 0000000..8c57283
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
@@ -0,0 +1,81 @@
+use anyhow::{bail, Result};
+use proxmox_installer_common::setup::SetupInfo;
+use serde::Serialize;
+use std::{collections::HashMap, fs, io, path::Path};
+
+use super::get_nic_list;
+
+const DMI_PATH: &str = "/sys/devices/virtual/dmi/id";
+
+pub fn get_sysinfo(pretty: bool) -> Result<String> {
+ let system_files = vec![
+ "product_serial",
+ "product_sku",
+ "product_uuid",
+ "product_name",
+ ];
+ let baseboard_files = vec!["board_asset_tag", "board_serial", "board_name"];
+ let chassis_files = vec!["chassis_serial", "chassis_sku", "chassis_asset_tag"];
+
+ let system = get_dmi_infos(system_files)?;
+ let baseboard = get_dmi_infos(baseboard_files)?;
+ let chassis = get_dmi_infos(chassis_files)?;
+
+ let mut mac_addresses: Vec<String> = Vec::new();
+ let links = get_nic_list()?;
+ for link in links {
+ let address = fs::read_to_string(format!("/sys/class/net/{link}/address"))?;
+ let address = String::from(address.trim());
+ mac_addresses.push(address);
+ }
+
+ let iso_info = Path::new("/run/proxmox-installer/iso-info.json");
+ let mut product = String::from("Not available. Would be one of the following: pve, pmg, pbs");
+ if iso_info.exists() {
+ let file = fs::File::open("/run/proxmox-installer/iso-info.json")?;
+ let reader = io::BufReader::new(file);
+ let setup_info: SetupInfo = serde_json::from_reader(reader)?;
+ product = setup_info.config.product.to_string();
+ }
+
+ let sysinfo = SysInfo {
+ product,
+ system,
+ baseboard,
+ chassis,
+ mac_addresses,
+ };
+ if pretty {
+ return Ok(serde_json::to_string_pretty(&sysinfo)?);
+ }
+ Ok(serde_json::to_string(&sysinfo)?)
+}
+
+#[derive(Debug, Serialize)]
+struct SysInfo {
+ product: String,
+ system: HashMap<String, String>,
+ baseboard: HashMap<String, String>,
+ chassis: HashMap<String, String>,
+ mac_addresses: Vec<String>,
+}
+
+fn get_dmi_infos(files: Vec<&str>) -> Result<HashMap<String, String>> {
+ let mut res: HashMap<String, String> = HashMap::new();
+
+ for file in files {
+ let path = format!("{DMI_PATH}/{file}");
+ let content = match fs::read_to_string(&path) {
+ Err(ref err) if err.kind() == std::io::ErrorKind::NotFound => continue,
+ Err(ref err) if err.kind() == std::io::ErrorKind::PermissionDenied => {
+ bail!("Could not read data. Are you running as root or with sudo?")
+ }
+ Err(err) => bail!("Error: '{err}' on '{path}'"),
+ Ok(content) => content.trim().into(),
+ };
+ let key = file.splitn(2, '_').last().unwrap();
+ res.insert(key.into(), content);
+ }
+
+ Ok(res)
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 22/30] auto-installer: helper: add subcommand to view indentifiers
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (20 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 21/30] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 23/30] auto-installer: fetch: add http post utility module Aaron Lauterer
` (9 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
It will collect the information from the current system and show the
payload of identifiers that will be send.
To avoid confusion, the subcommands for the device info and filter
matching have been renamed.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/bin/proxmox-autoinst-helper.rs | 54 +++++++++----------
1 file changed, 26 insertions(+), 28 deletions(-)
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
index 058d5ff..f0ee8f4 100644
--- a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
+++ b/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
@@ -2,12 +2,13 @@ use anyhow::{bail, Result};
use clap::{Args, Parser, Subcommand, ValueEnum};
use glob::Pattern;
use regex::Regex;
-use serde::{Deserialize, Serialize};
+use serde::Serialize;
use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
+ fetch_plugins::utils::{sysinfo, get_nic_list},
utils::{get_matched_udev_indexes, get_single_udev_index},
};
@@ -23,13 +24,14 @@ struct Cli {
#[derive(Subcommand, Debug)]
enum Commands {
ValidateAnswer(CommandValidateAnswer),
- Match(CommandMatch),
- Info(CommandInfo),
+ DeviceMatch(CommandDeviceMatch),
+ DeviceInfo(CommandDeviceInfo),
+ Identifiers(CommandIdentifiers),
}
/// Show device information that can be used for filters
#[derive(Args, Debug)]
-struct CommandInfo {
+struct CommandDeviceInfo {
/// For which device type information should be shown
#[arg(name="type", short, long, value_enum, default_value_t=AllDeviceTypes::All)]
device: AllDeviceTypes,
@@ -52,7 +54,7 @@ struct CommandInfo {
/// proxmox-autoinst-helper match --filter-match all disk 'ID_SERIAL_SHORT=*2222*' 'DEVNAME=*nvme*'
#[derive(Args, Debug)]
#[command(verbatim_doc_comment)]
-struct CommandMatch {
+struct CommandDeviceMatch {
/// Device type to match the filter against
r#type: Devicetype,
@@ -74,6 +76,11 @@ struct CommandValidateAnswer {
debug: bool,
}
+/// Show identifiers for the current machine. This information is part of the POST request to fetch
+/// an answer file.
+#[derive(Args, Debug)]
+struct CommandIdentifiers {}
+
#[derive(Args, Debug)]
struct GlobalOpts {
/// Output format
@@ -109,9 +116,10 @@ struct Devs {
fn main() {
let args = Cli::parse();
let res = match &args.command {
- Commands::Info(args) => info(args),
- Commands::Match(args) => match_filter(args),
Commands::ValidateAnswer(args) => validate_answer(args),
+ Commands::DeviceInfo(args) => info(args),
+ Commands::DeviceMatch(args) => match_filter(args),
+ Commands::Identifiers(args) => show_identifiers(args),
};
if let Err(err) = res {
eprintln!("{err}");
@@ -119,7 +127,7 @@ fn main() {
}
}
-fn info(args: &CommandInfo) -> Result<()> {
+fn info(args: &CommandDeviceInfo) -> Result<()> {
let mut devs = Devs {
disks: None,
nics: None,
@@ -141,7 +149,7 @@ fn info(args: &CommandInfo) -> Result<()> {
Ok(())
}
-fn match_filter(args: &CommandMatch) -> Result<()> {
+fn match_filter(args: &CommandDeviceMatch) -> Result<()> {
let devs: BTreeMap<String, BTreeMap<String, String>> = match args.r#type {
Devicetype::Disk => get_disks().unwrap(),
Devicetype::Network => get_nics().unwrap(),
@@ -205,6 +213,14 @@ fn validate_answer(args: &CommandValidateAnswer) -> Result<()> {
Ok(())
}
+fn show_identifiers(_args: &CommandIdentifiers) -> Result<()> {
+ match sysinfo::get_sysinfo(true) {
+ Ok(res) => println!("{res}"),
+ Err(err) => eprintln!("Error fetching system identifiers: {err}"),
+ }
+ Ok(())
+}
+
fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let unwantend_block_devs = vec![
"ram[0-9]*",
@@ -275,30 +291,12 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
Ok(disks)
}
-#[derive(Deserialize, Debug)]
-struct IpLinksInfo {
- ifname: String,
-}
fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
- let ip_output = Command::new("/usr/sbin/ip")
- .arg("-j")
- .arg("link")
- .output()?;
- let parsed_links: Vec<IpLinksInfo> =
- serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
- let mut links: Vec<String> = Vec::new();
-
- for link in parsed_links {
- if link.ifname == *"lo" {
- continue;
- }
- links.push(link.ifname);
- }
-
+ let links = get_nic_list()?;
for link in links {
let path = format!("/sys/class/net/{link}");
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 23/30] auto-installer: fetch: add http post utility module
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (21 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 22/30] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 24/30] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
` (8 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
It sends a http(s) POST request with the sysinfo as payload and expects
an answer file in return.
In order to handle non FQDN URLs (e.g. IP addresses) and self signed
certificates, it can optionally take an SHA256 fingerprint of the
certificate. This can of course also be used to pin a certificate
explicitly, even if it would be in the trust chain.
A custom cert verifier for ureq / rustl was necessary to get cert
fingerprint matching to work.
If no fingerprint is proviced, we switch rustls to native-certs and
native-tls.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-auto-installer/Cargo.toml | 6 ++
.../src/fetch_plugins/utils/mod.rs | 1 +
.../src/fetch_plugins/utils/post.rs | 94 +++++++++++++++++++
3 files changed, 101 insertions(+)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/utils/post.rs
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index bb0b49c..ac2f3a6 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -18,3 +18,9 @@ toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
regex = "1.7"
+ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
+rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
+rustls-native-certs = "0.6"
+native-tls = "0.2"
+sha2 = "0.10"
+hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
index b3e9dad..6b4c7db 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
@@ -12,6 +12,7 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
+pub mod post;
pub mod sysinfo;
/// Searches for upper and lower case existence of the partlabel in the search_path
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
new file mode 100644
index 0000000..193e920
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
@@ -0,0 +1,94 @@
+use anyhow::Result;
+use rustls::ClientConfig;
+use sha2::{Digest, Sha256};
+use std::sync::Arc;
+use ureq::{Agent, AgentBuilder};
+
+/// Issues a POST request with the payload (JSON). Optionally a SHA256 fingerprint can be used to
+/// check the cert against it, instead of the regular cert validation.
+/// To gather the sha256 fingerprint you can use the following command:
+/// ```no_compile
+/// openssl s_client -connect <host>:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin
+/// ```
+///
+/// # Arguemnts
+/// * `url` - URL to call
+/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional.
+/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string.
+pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> {
+ let answer ;
+
+ if let Some(fingerprint) = fingerprint {
+ let tls_config = ClientConfig::builder()
+ .with_safe_defaults()
+ .with_custom_certificate_verifier(VerifyCertFingerprint::new(fingerprint)?)
+ .with_no_client_auth();
+
+ let agent: Agent = AgentBuilder::new().tls_config(Arc::new(tls_config)).build();
+
+ answer = agent
+ .post(&url)
+ .set("Content-type", "application/json; charset=utf-")
+ .send_string(&payload)?
+ .into_string()?;
+ } else {
+ let mut roots = rustls::RootCertStore::empty();
+ for cert in rustls_native_certs::load_native_certs()? {
+ roots.add(&rustls::Certificate(cert.0)).unwrap();
+ }
+
+ let tls_config = rustls::ClientConfig::builder()
+ .with_safe_defaults()
+ .with_root_certificates(roots)
+ .with_no_client_auth();
+
+ let agent = AgentBuilder::new()
+ .tls_connector(Arc::new(native_tls::TlsConnector::new()?))
+ .tls_config(Arc::new(tls_config))
+ .build();
+ answer = agent
+ .post(&url)
+ .set("Content-type", "application/json; charset=utf-")
+ .timeout(std::time::Duration::from_secs(60))
+ .send_string(&payload)?
+ .into_string()?;
+ }
+ Ok(answer)
+}
+
+struct VerifyCertFingerprint {
+ cert_fingerprint: Vec<u8>,
+}
+
+impl VerifyCertFingerprint {
+ fn new<S: AsRef<str>>(cert_fingerprint: S) -> Result<std::sync::Arc<Self>> {
+ let cert_fingerprint = cert_fingerprint.as_ref();
+ let sanitized = cert_fingerprint.replace(':', "");
+ let decoded = hex::decode(sanitized)?;
+ Ok(std::sync::Arc::new(Self {
+ cert_fingerprint: decoded,
+ }))
+ }
+}
+
+impl rustls::client::ServerCertVerifier for VerifyCertFingerprint {
+ fn verify_server_cert(
+ &self,
+ end_entity: &rustls::Certificate,
+ _intermediates: &[rustls::Certificate],
+ _server_name: &rustls::ServerName,
+ _scts: &mut dyn Iterator<Item = &[u8]>,
+ _ocsp_response: &[u8],
+ _now: std::time::SystemTime,
+ ) -> Result<rustls::client::ServerCertVerified, rustls::Error> {
+ let mut hasher = Sha256::new();
+ hasher.update(end_entity);
+ let result = hasher.finalize();
+
+ if result.as_slice() == self.cert_fingerprint {
+ Ok(rustls::client::ServerCertVerified::assertion())
+ } else {
+ Err(rustls::Error::General("Fingerprint did not match!".into()))
+ }
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 24/30] auto-installer: fetch: add http plugin to fetch answer
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (22 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 23/30] auto-installer: fetch: add http post utility module Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 25/30] control: update build depends for auto installer Aaron Lauterer
` (7 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
This plugin will send a HTTP POST request with identifying sysinfo to
fetch an answer file. The provided sysinfo can be used to identify the
system and generate a matching answer file on demand.
The URL to send the request to, can be defined in two ways. Via a custom
DHCP option or a TXT record on a predefined subdomain, relative to the
search domain received via DHCP.
Additionally it is possible to specify a SHA256 SSL fingerprint. This
can be useful if a self-signed certificate is used or the URL is using
an IP address instead of an FQDN. Even with a trusted cert, it can be
used to pin this specific certificate.
The certificate fingerprint can either be placed on the `proxmoxinst`
partition and needs to be called `cert_fingerprint.txt`, or it can be
provided in a second custom DHCP option or a TXT record.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.../src/bin/proxmox-fetch-answer.rs | 11 +-
.../src/fetch_plugins/http.rs | 190 ++++++++++++++++++
.../src/fetch_plugins/mod.rs | 1 +
unconfigured.sh | 9 +
4 files changed, 208 insertions(+), 3 deletions(-)
create mode 100644 proxmox-auto-installer/src/fetch_plugins/http.rs
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
index a3681a2..6d42df2 100644
--- a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
+++ b/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
@@ -1,6 +1,9 @@
use anyhow::{anyhow, Error, Result};
use log::{error, info, LevelFilter};
-use proxmox_auto_installer::{fetch_plugins::partition::FetchFromPartition, log::AutoInstLogger};
+use proxmox_auto_installer::{
+ fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition},
+ log::AutoInstLogger,
+};
use std::io::Write;
use std::process::{Command, ExitCode, Stdio};
@@ -18,8 +21,10 @@ fn fetch_answer() -> Result<String> {
Ok(answer) => return Ok(answer),
Err(err) => info!("Fetching answer file from partition failed: {err}"),
}
- // TODO: add more options to get an answer file, e.g. download from url where url could be
- // fetched via txt records on predefined subdomain, kernel param, dhcp option, ...
+ match FetchFromHTTP::get_answer() {
+ Ok(answer) => return Ok(answer),
+ Err(err) => info!("Fetching answer file via HTTP failed: {err}"),
+ }
Err(Error::msg("Could not find any answer file!"))
}
diff --git a/proxmox-auto-installer/src/fetch_plugins/http.rs b/proxmox-auto-installer/src/fetch_plugins/http.rs
new file mode 100644
index 0000000..4ac9afb
--- /dev/null
+++ b/proxmox-auto-installer/src/fetch_plugins/http.rs
@@ -0,0 +1,190 @@
+use anyhow::{bail, Error, Result};
+use log::info;
+use std::{
+ fs::{self, read_to_string},
+ path::Path,
+ process::Command,
+};
+
+use crate::fetch_plugins::utils::{post, sysinfo};
+
+use super::utils;
+
+static CERT_FINGERPRINT_FILE: &str = "cert_fingerprint.txt";
+static ANSWER_SUBDOMAIN: &str = "proxmoxinst";
+static ANSWER_SUBDOMAIN_FP: &str = "proxmoxinst-fp";
+
+// It is possible to set custom DHPC options. Option numbers 224 to 254 [0].
+// To use them with dhclient, we need to configure it to request them and what they should be
+// called.
+//
+// e.g. /etc/dhcp/dhclient.conf:
+// ```
+// option proxmoxinst-url code 250 = text;
+// option proxmoxinst-fp code 251 = text;
+// also request proxmoxinst-url, proxmoxinst-fp;
+// ```
+//
+// The results will end up in the /var/lib/dhcp/dhclient.leases file from where we can fetch them
+//
+// [0] https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml
+static DHCP_URL_OPTION: &str = "proxmoxinst-url";
+static DHCP_FP_OPTION: &str = "proxmoxinst-fp";
+static DHCP_LEASE_FILE: &str = "/var/lib/dhcp/dhclient.leases";
+
+pub struct FetchFromHTTP;
+
+impl FetchFromHTTP {
+ /// Will try to fetch the answer.toml by sending a HTTP POST request. The URL can be configured
+ /// either via DHCP or DNS.
+ /// DHCP options are checked first. The SSL certificate need to be either trusted by the root
+ /// certs or a SHA256 fingerprint needs to be provided. The SHA256 SSL fingerprint can either
+ /// be placed in a `cert_fingerprint.txt` file in the `proxmoxinst` partition, as DHCP option,
+ /// or as DNS TXT record. If provided, the `cert_fingerprint.txt` file has preference.
+ pub fn get_answer() -> Result<String> {
+ info!("Checking for certificate fingerprint in file.");
+ let mut fingerprint: Option<String> = match Self::get_cert_fingerprint_from_file() {
+ Ok(fp) => Some(fp),
+ Err(err) => {
+ info!("{err}");
+ None
+ }
+ };
+
+ let answer_url: String;
+
+ (answer_url, fingerprint) = match Self::fetch_dhcp(fingerprint.clone()) {
+ Ok((url, fp)) => (url, fp),
+ Err(err) => {
+ info!("{err}");
+ Self::fetch_dns(fingerprint.clone())?
+ }
+ };
+
+ if fingerprint.is_some() {
+ let fp = fingerprint.clone();
+ fs::write("/tmp/cert_fingerprint", fp.unwrap()).ok();
+ }
+
+ info!("Gathering system information.");
+ let payload = sysinfo::get_sysinfo(false)?;
+ info!("Sending POST request to '{answer_url}'.");
+ let answer = post::call(answer_url, fingerprint.as_deref(), payload)?;
+ Ok(answer)
+ }
+
+ /// Reads certificate fingerprint from file
+ pub fn get_cert_fingerprint_from_file() -> Result<String> {
+ let mount_path = utils::mount_proxmoxinst_part()?;
+ let cert_path = Path::new(mount_path.as_str()).join(CERT_FINGERPRINT_FILE);
+ match cert_path.try_exists() {
+ Ok(true) => {
+ info!("Found certifacte fingerprint file.");
+ Ok(fs::read_to_string(cert_path)?.trim().into())
+ }
+ _ => Err(Error::msg(format!(
+ "could not find cert fingerprint file expected at: {}",
+ cert_path.display()
+ ))),
+ }
+ }
+
+ /// Fetches search domain from resolv.conf file
+ fn get_search_domain() -> Result<String> {
+ info!("Retrieving default search domain.");
+ for line in read_to_string("/etc/resolv.conf")?.lines() {
+ if let Some((key, value)) = line.split_once(' ') {
+ if key == "search" {
+ return Ok(value.trim().into());
+ }
+ }
+ }
+ Err(Error::msg("Could not find search domain in resolv.conf."))
+ }
+
+ /// Runs a TXT DNS query on the domain provided
+ fn query_txt_record(query: String) -> Result<String> {
+ info!("Querying TXT record for '{query}'");
+ let url: String;
+ match Command::new("dig")
+ .args(["txt", "+short"])
+ .arg(&query)
+ .output()
+ {
+ Ok(output) => {
+ if output.status.success() {
+ url = String::from_utf8(output.stdout)?
+ .replace('"', "")
+ .trim()
+ .into();
+ if url.is_empty() {
+ bail!("Got empty response.");
+ }
+ } else {
+ bail!(
+ "Error querying DNS record '{query}' : {}",
+ String::from_utf8(output.stderr)?
+ );
+ }
+ }
+ Err(err) => bail!("Error querying DNS record '{query}': {err}"),
+ }
+ info!("Found: '{url}'");
+ Ok(url)
+ }
+
+ /// Tries to fetch answer URL and SSL fingerprint info from DNS
+ fn fetch_dns(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> {
+ let search_domain = Self::get_search_domain()?;
+
+ let answer_url = match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN}.{search_domain}"))
+ {
+ Ok(url) => url,
+ Err(err) => bail!("{err}"),
+ };
+
+ if fingerprint.is_none() {
+ fingerprint =
+ match Self::query_txt_record(format!("{ANSWER_SUBDOMAIN_FP}.{search_domain}")) {
+ Ok(fp) => Some(fp),
+ Err(err) => {
+ info!("{err}");
+ None
+ }
+ };
+ }
+ Ok((answer_url, fingerprint))
+ }
+
+ /// Tries to fetch answer URL and SSL fingerprint info from DHCP options
+ fn fetch_dhcp(mut fingerprint: Option<String>) -> Result<(String, Option<String>)> {
+ let leases = fs::read_to_string(DHCP_LEASE_FILE)?;
+
+ let mut answer_url: Option<String> = None;
+
+ let url_match = format!("option {DHCP_URL_OPTION}");
+ let fp_match = format!("option {DHCP_FP_OPTION}");
+
+ for line in leases.lines() {
+ if answer_url.is_none() && line.trim().starts_with(url_match.as_str()) {
+ answer_url = Self::strip_dhcp_option(line.split(' ').nth_back(0));
+ }
+ if fingerprint.is_none() && line.trim().starts_with(fp_match.as_str()) {
+ fingerprint = Self::strip_dhcp_option(line.split(' ').nth_back(0));
+ }
+ }
+
+ let answer_url = match answer_url {
+ None => bail!("No DHCP option found for fetch URL."),
+ Some(url) => url,
+ };
+
+ Ok((answer_url, fingerprint))
+ }
+
+ /// Clean DHCP option string
+ fn strip_dhcp_option(value: Option<&str>) -> Option<String> {
+ // value is expected to be in format: "value";
+ value.map(|value| String::from(&value[1..value.len() - 2]))
+ }
+}
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
index 6f1e8a2..354fa7e 100644
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ b/proxmox-auto-installer/src/fetch_plugins/mod.rs
@@ -1,2 +1,3 @@
+pub mod http;
pub mod partition;
pub mod utils;
diff --git a/unconfigured.sh b/unconfigured.sh
index f02336a..dbdb027 100755
--- a/unconfigured.sh
+++ b/unconfigured.sh
@@ -212,6 +212,15 @@ if [ $proxdebug -ne 0 ]; then
debugsh || true
fi
+# add custom DHCP options for auto installer
+if [ $proxauto -ne 0 ]; then
+ cat >> /etc/dhcp/dhclient.conf <<EOF
+option proxmoxinst-url code 250 = text;
+option proxmoxinst-fp code 251 = text;
+also request proxmoxinst-url, proxmoxinst-fp;
+EOF
+fi
+
# try to get ip config with dhcp
echo -n "Attempting to get DHCP leases... "
dhclient -v
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 25/30] control: update build depends for auto installer
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (23 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 24/30] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 26/30] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
` (6 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
debian/control | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/debian/control b/debian/control
index 3ca208b..1326400 100644
--- a/debian/control
+++ b/debian/control
@@ -8,10 +8,20 @@ Build-Depends: cargo:native,
libgtk3-perl,
libpve-common-perl,
librsvg2-bin,
+ librust-anyhow-dev,
+ librust-clap-dev,
librust-cursive+termion-backend-dev (>= 0.20.0),
+ librust-glob-dev,
+ librust-hex-dev,
+ librust-native-tls-dev,
librust-regex-1+default-dev (>= 1.7~~),
+ librust-rustls-dev,
+ librust-rustls-native-certs-dev,
librust-serde-1+default-dev,
librust-serde-json-1+default-dev,
+ librust-sha2-dev,
+ librust-ureq-dev,
+ librust-toml-dev,
libtest-mockmodule-perl,
perl,
rustc:native,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 26/30] auto installer: factor out fetch-answer and autoinst-helper
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (24 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 25/30] control: update build depends for auto installer Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 27/30] low-level: write low level config to /tmp Aaron Lauterer
` (5 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
Putting proxmox-fetch-answer into it's own crate, will keep the use of
OpenSSL localized to where we need it. Otherwise building other binaries
will always depend on OpenSSL as well, even without actually needing it.
Having a dedicated crate for the proxmox-autoinst-helper should make it
easier to build it independently to have it available outside of the
install environment.
The fetch plugins have been moved to the proxmox-fetch-answer crate,
except for the 'get_nic_list' function and 'sysinfo.rs'. Since both are
also needed by the proxmox-autoinst-helper, they are kept in the
proxmox-auto-installer crate.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 2 ++
Makefile | 5 +++-
proxmox-auto-installer/Cargo.toml | 6 ----
.../src/fetch_plugins/mod.rs | 3 --
proxmox-auto-installer/src/lib.rs | 2 +-
.../src/{fetch_plugins/utils => }/sysinfo.rs | 2 +-
proxmox-auto-installer/src/utils.rs | 25 ++++++++++++++++
proxmox-autoinst-helper/Cargo.toml | 21 +++++++++++++
.../src/main.rs | 5 ++--
proxmox-fetch-answer/Cargo.toml | 22 ++++++++++++++
.../src/fetch_plugins/http.rs | 3 +-
proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 ++
.../src/fetch_plugins/partition.rs | 2 +-
.../src/fetch_plugins/utils/mod.rs | 30 +------------------
.../src/fetch_plugins/utils/post.rs | 2 +-
.../src/main.rs | 8 ++---
16 files changed, 90 insertions(+), 51 deletions(-)
delete mode 100644 proxmox-auto-installer/src/fetch_plugins/mod.rs
rename proxmox-auto-installer/src/{fetch_plugins/utils => }/sysinfo.rs (98%)
create mode 100644 proxmox-autoinst-helper/Cargo.toml
rename proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs => proxmox-autoinst-helper/src/main.rs (98%)
create mode 100644 proxmox-fetch-answer/Cargo.toml
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/http.rs (98%)
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/partition.rs (100%)
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/utils/mod.rs (81%)
rename {proxmox-auto-installer => proxmox-fetch-answer}/src/fetch_plugins/utils/post.rs (99%)
rename proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs => proxmox-fetch-answer/src/main.rs (93%)
diff --git a/Cargo.toml b/Cargo.toml
index 7017ac5..b694d5b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,8 @@
[workspace]
members = [
"proxmox-auto-installer",
+ "proxmox-autoinst-helper",
+ "proxmox-fetch-answer",
"proxmox-installer-common",
"proxmox-tui-installer",
]
diff --git a/Makefile b/Makefile
index 197a351..e32d28f 100644
--- a/Makefile
+++ b/Makefile
@@ -52,6 +52,8 @@ $(BUILDDIR):
proxinstall \
proxmox-low-level-installer \
proxmox-auto-installer/ \
+ proxmox-autoinst-helper/ \
+ proxmox-fetch-answer/ \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
@@ -124,7 +126,8 @@ $(COMPILED_BINS): cargo-build
cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
- --bin proxmox-fetch-answer --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
+ --package proxmox-fetch-answer --bin proxmox-fetch-answer \
+ --package proxmox-autoinst-helper --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml
index ac2f3a6..bb0b49c 100644
--- a/proxmox-auto-installer/Cargo.toml
+++ b/proxmox-auto-installer/Cargo.toml
@@ -18,9 +18,3 @@ toml = "0.7"
enum-iterator = "0.6.0"
log = "0.4.20"
regex = "1.7"
-ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
-rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
-rustls-native-certs = "0.6"
-native-tls = "0.2"
-sha2 = "0.10"
-hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/mod.rs b/proxmox-auto-installer/src/fetch_plugins/mod.rs
deleted file mode 100644
index 354fa7e..0000000
--- a/proxmox-auto-installer/src/fetch_plugins/mod.rs
+++ /dev/null
@@ -1,3 +0,0 @@
-pub mod http;
-pub mod partition;
-pub mod utils;
diff --git a/proxmox-auto-installer/src/lib.rs b/proxmox-auto-installer/src/lib.rs
index 0a153b2..3bdf0b5 100644
--- a/proxmox-auto-installer/src/lib.rs
+++ b/proxmox-auto-installer/src/lib.rs
@@ -1,5 +1,5 @@
pub mod answer;
-pub mod fetch_plugins;
pub mod log;
+pub mod sysinfo;
pub mod udevinfo;
pub mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs b/proxmox-auto-installer/src/sysinfo.rs
similarity index 98%
rename from proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
rename to proxmox-auto-installer/src/sysinfo.rs
index 8c57283..f5de214 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/sysinfo.rs
+++ b/proxmox-auto-installer/src/sysinfo.rs
@@ -3,7 +3,7 @@ use proxmox_installer_common::setup::SetupInfo;
use serde::Serialize;
use std::{collections::HashMap, fs, io, path::Path};
-use super::get_nic_list;
+use crate::utils::get_nic_list;
const DMI_PATH: &str = "/sys/devices/virtual/dmi/id";
diff --git a/proxmox-auto-installer/src/utils.rs b/proxmox-auto-installer/src/utils.rs
index ea645ad..ff90ae8 100644
--- a/proxmox-auto-installer/src/utils.rs
+++ b/proxmox-auto-installer/src/utils.rs
@@ -72,6 +72,31 @@ pub fn get_single_udev_index(
Ok(dev_index.unwrap())
}
+#[derive(Deserialize, Debug)]
+struct IpLinksUdevInfo {
+ ifname: String,
+}
+
+/// Returns vec of usable NICs
+pub fn get_nic_list() -> Result<Vec<String>> {
+ let ip_output = Command::new("/usr/sbin/ip")
+ .arg("-j")
+ .arg("link")
+ .output()?;
+ let parsed_links: Vec<IpLinksUdevInfo> =
+ serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
+ let mut links: Vec<String> = Vec::new();
+
+ for link in parsed_links {
+ if link.ifname == *"lo" {
+ continue;
+ }
+ links.push(link.ifname);
+ }
+
+ Ok(links)
+}
+
pub fn get_matched_udev_indexes(
filter: BTreeMap<String, String>,
udev_list: &BTreeMap<String, BTreeMap<String, String>>,
diff --git a/proxmox-autoinst-helper/Cargo.toml b/proxmox-autoinst-helper/Cargo.toml
new file mode 100644
index 0000000..2a88c0f
--- /dev/null
+++ b/proxmox-autoinst-helper/Cargo.toml
@@ -0,0 +1,21 @@
+[package]
+name = "proxmox-autoinst-helper"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+glob = "0.3"
+proxmox-auto-installer = { path = "../proxmox-auto-installer" }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+toml = "0.7"
+log = "0.4.20"
+regex = "1.7"
diff --git a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs b/proxmox-autoinst-helper/src/main.rs
similarity index 98%
rename from proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
rename to proxmox-autoinst-helper/src/main.rs
index f0ee8f4..fe1cbec 100644
--- a/proxmox-auto-installer/src/bin/proxmox-autoinst-helper.rs
+++ b/proxmox-autoinst-helper/src/main.rs
@@ -8,8 +8,8 @@ use std::{collections::BTreeMap, fs, io::Read, path::PathBuf, process::Command};
use proxmox_auto_installer::{
answer::Answer,
answer::FilterMatch,
- fetch_plugins::utils::{sysinfo, get_nic_list},
- utils::{get_matched_udev_indexes, get_single_udev_index},
+ sysinfo,
+ utils::{get_matched_udev_indexes, get_nic_list, get_single_udev_index},
};
/// This tool validates the format of an answer file. Additionally it can test match filters and
@@ -291,7 +291,6 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
Ok(disks)
}
-
fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
diff --git a/proxmox-fetch-answer/Cargo.toml b/proxmox-fetch-answer/Cargo.toml
new file mode 100644
index 0000000..fbcca46
--- /dev/null
+++ b/proxmox-fetch-answer/Cargo.toml
@@ -0,0 +1,22 @@
+[package]
+name = "proxmox-fetch-answer"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+proxmox-auto-installer = { path = "../proxmox-auto-installer" }
+log = "0.4.20"
+ureq = { version = "2.6", features = [ "native-certs", "native-tls" ] }
+rustls = { version = "0.20", features = [ "dangerous_configuration" ] }
+rustls-native-certs = "0.6"
+native-tls = "0.2"
+sha2 = "0.10"
+hex = "0.4"
diff --git a/proxmox-auto-installer/src/fetch_plugins/http.rs b/proxmox-fetch-answer/src/fetch_plugins/http.rs
similarity index 98%
rename from proxmox-auto-installer/src/fetch_plugins/http.rs
rename to proxmox-fetch-answer/src/fetch_plugins/http.rs
index 4ac9afb..5772c42 100644
--- a/proxmox-auto-installer/src/fetch_plugins/http.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/http.rs
@@ -6,7 +6,8 @@ use std::{
process::Command,
};
-use crate::fetch_plugins::utils::{post, sysinfo};
+use crate::fetch_plugins::utils::post;
+use proxmox_auto_installer::sysinfo;
use super::utils;
diff --git a/proxmox-fetch-answer/src/fetch_plugins/mod.rs b/proxmox-fetch-answer/src/fetch_plugins/mod.rs
new file mode 100644
index 0000000..d148e4c
--- /dev/null
+++ b/proxmox-fetch-answer/src/fetch_plugins/mod.rs
@@ -0,0 +1,3 @@
+pub(crate) mod http;
+pub(crate) mod partition;
+pub(crate) mod utils;
diff --git a/proxmox-auto-installer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
similarity index 100%
rename from proxmox-auto-installer/src/fetch_plugins/partition.rs
rename to proxmox-fetch-answer/src/fetch_plugins/partition.rs
index 0c47a62..dbe5dda 100644
--- a/proxmox-auto-installer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -1,6 +1,6 @@
use anyhow::{Error, Result};
-use std::{fs::read_to_string, path::Path};
use log::info;
+use std::{fs::read_to_string, path::Path};
use crate::fetch_plugins::utils::mount_proxmoxinst_part;
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
similarity index 81%
rename from proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
rename to proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
index 6b4c7db..e5ea4b8 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/mod.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
@@ -1,7 +1,5 @@
use anyhow::{Error, Result};
use log::{info, warn};
-use serde::Deserialize;
-use serde_json;
use std::{
fs::{self, create_dir_all},
path::{Path, PathBuf},
@@ -12,8 +10,7 @@ static ANSWER_MP: &str = "/mnt/answer";
static PARTLABEL: &str = "proxmoxinst";
static SEARCH_PATH: &str = "/dev/disk/by-label";
-pub mod post;
-pub mod sysinfo;
+pub(crate) mod post;
/// Searches for upper and lower case existence of the partlabel in the search_path
///
@@ -86,28 +83,3 @@ fn check_if_mounted(target_path: &str) -> Result<bool> {
}
Ok(false)
}
-
-#[derive(Deserialize, Debug)]
-struct IpLinksUdevInfo {
- ifname: String,
-}
-
-/// Returns vec of usable NICs
-pub fn get_nic_list() -> Result<Vec<String>> {
- let ip_output = Command::new("/usr/sbin/ip")
- .arg("-j")
- .arg("link")
- .output()?;
- let parsed_links: Vec<IpLinksUdevInfo> =
- serde_json::from_str(String::from_utf8(ip_output.stdout)?.as_str())?;
- let mut links: Vec<String> = Vec::new();
-
- for link in parsed_links {
- if link.ifname == *"lo" {
- continue;
- }
- links.push(link.ifname);
- }
-
- Ok(links)
-}
diff --git a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs b/proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
similarity index 99%
rename from proxmox-auto-installer/src/fetch_plugins/utils/post.rs
rename to proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
index 193e920..89658f3 100644
--- a/proxmox-auto-installer/src/fetch_plugins/utils/post.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
@@ -16,7 +16,7 @@ use ureq::{Agent, AgentBuilder};
/// * `fingerprint` - SHA256 cert fingerprint if certificate pinning should be used. Optional.
/// * `payload` - The payload to send to the server. Expected to be a JSON formatted string.
pub fn call(url: String, fingerprint: Option<&str>, payload: String) -> Result<String> {
- let answer ;
+ let answer;
if let Some(fingerprint) = fingerprint {
let tls_config = ClientConfig::builder()
diff --git a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs b/proxmox-fetch-answer/src/main.rs
similarity index 93%
rename from proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
rename to proxmox-fetch-answer/src/main.rs
index 6d42df2..8c762e9 100644
--- a/proxmox-auto-installer/src/bin/proxmox-fetch-answer.rs
+++ b/proxmox-fetch-answer/src/main.rs
@@ -1,12 +1,12 @@
use anyhow::{anyhow, Error, Result};
+use fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition};
use log::{error, info, LevelFilter};
-use proxmox_auto_installer::{
- fetch_plugins::{http::FetchFromHTTP, partition::FetchFromPartition},
- log::AutoInstLogger,
-};
+use proxmox_auto_installer::log::AutoInstLogger;
use std::io::Write;
use std::process::{Command, ExitCode, Stdio};
+mod fetch_plugins;
+
static LOGGER: AutoInstLogger = AutoInstLogger;
pub fn init_log() -> Result<()> {
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 27/30] low-level: write low level config to /tmp
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (25 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 26/30] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
@ 2024-04-04 14:48 ` Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 28/30] common: add deserializer for FsType Aaron Lauterer
` (4 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:48 UTC (permalink / raw)
To: pve-devel
This helps to know how the system was set up in steps after the
installation. For example in debug mode or when using post commands in
the automatic/unattended installation.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-low-level-installer | 1 +
1 file changed, 1 insertion(+)
diff --git a/proxmox-low-level-installer b/proxmox-low-level-installer
index 54f689a..935bf17 100755
--- a/proxmox-low-level-installer
+++ b/proxmox-low-level-installer
@@ -69,6 +69,7 @@ sub read_and_merge_config {
Proxmox::Install::Config::merge($config);
log_info("got installation config: ". to_json(Proxmox::Install::Config::get(), { utf8 => 1, canonical => 1 }) ."\n");
+ file_write_all("/tmp/low-level-config.json", to_json(Proxmox::Install::Config::get()));
}
sub send_reboot_ui_message {
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 28/30] common: add deserializer for FsType
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (26 preceding siblings ...)
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 27/30] low-level: write low level config to /tmp Aaron Lauterer
@ 2024-04-04 14:49 ` Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 29/30] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
` (3 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:49 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/Cargo.toml | 1 +
proxmox-installer-common/src/options.rs | 10 ++++++---
proxmox-installer-common/src/setup.rs | 30 ++++++++++++++++++++++---
3 files changed, 35 insertions(+), 6 deletions(-)
diff --git a/proxmox-installer-common/Cargo.toml b/proxmox-installer-common/Cargo.toml
index bde5457..70f828a 100644
--- a/proxmox-installer-common/Cargo.toml
+++ b/proxmox-installer-common/Cargo.toml
@@ -8,5 +8,6 @@ exclude = [ "build", "debian" ]
homepage = "https://www.proxmox.com"
[dependencies]
+regex = "1.7"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
diff --git a/proxmox-installer-common/src/options.rs b/proxmox-installer-common/src/options.rs
index 1efac66..1e782f9 100644
--- a/proxmox-installer-common/src/options.rs
+++ b/proxmox-installer-common/src/options.rs
@@ -1,6 +1,6 @@
+use serde::Deserialize;
use std::net::{IpAddr, Ipv4Addr};
use std::{cmp, fmt};
-use serde::Deserialize;
use crate::setup::{
LocaleInfo, NetworkInfo, ProductConfig, ProxmoxProduct, RuntimeInfo, SetupInfo,
@@ -32,8 +32,11 @@ pub enum ZfsRaidLevel {
Raid0,
Raid1,
Raid10,
+ #[serde(rename = "raidz-1")]
RaidZ,
+ #[serde(rename = "raidz-2")]
RaidZ2,
+ #[serde(rename = "raidz-3")]
RaidZ3,
}
@@ -51,7 +54,8 @@ impl fmt::Display for ZfsRaidLevel {
}
}
-#[derive(Copy, Clone, Debug, Eq, PartialEq)]
+#[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(rename_all = "lowercase")]
pub enum FsType {
Ext4,
Xfs,
@@ -226,7 +230,7 @@ pub enum AdvancedBootdiskOptions {
Btrfs(BtrfsBootdiskOptions),
}
-#[derive(Clone, Debug, PartialEq)]
+#[derive(Clone, Debug, Deserialize, PartialEq)]
pub struct Disk {
pub index: String,
pub path: String,
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index 25d0e9e..c580477 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -155,7 +155,7 @@ pub fn installer_setup(in_test_mode: bool) -> Result<(SetupInfo, LocaleInfo, Run
}
}
-#[derive(Serialize)]
+#[derive(Debug, Deserialize, Serialize)]
pub struct InstallZfsOption {
pub ashift: usize,
#[serde(serialize_with = "serialize_as_display")]
@@ -402,11 +402,11 @@ pub fn spawn_low_level_installer(test_mode: bool) -> io::Result<process::Child>
}
/// See Proxmox::Install::Config
-#[derive(Serialize)]
+#[derive(Debug, Deserialize, Serialize)]
pub struct InstallConfig {
pub autoreboot: usize,
- #[serde(serialize_with = "serialize_fstype")]
+ #[serde(serialize_with = "serialize_fstype", deserialize_with = "deserialize_fs_type")]
pub filesys: FsType,
pub hdsize: f64,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -481,3 +481,27 @@ where
serializer.collect_str(value)
}
+
+pub fn deserialize_fs_type<'de, D>(deserializer: D) -> Result<FsType, D::Error>
+where
+ D: Deserializer<'de>,
+{
+ use FsType::*;
+ let de_fs: String = Deserialize::deserialize(deserializer)?;
+
+ println!("deserializing fstype");
+ match de_fs.as_str() {
+ "ext4" => Ok(Ext4),
+ "xfs" => Ok(Xfs),
+ "zfs (RAID0)" => Ok(Zfs(ZfsRaidLevel::Raid0)),
+ "zfs (RAID1)" => Ok(Zfs(ZfsRaidLevel::Raid1)),
+ "zfs (RAID10)" => Ok(Zfs(ZfsRaidLevel::Raid10)),
+ "zfs (RAIDZ-1)" => Ok(Zfs(ZfsRaidLevel::RaidZ)),
+ "zfs (RAIDZ-2)" => Ok(Zfs(ZfsRaidLevel::RaidZ2)),
+ "zfs (RAIDZ-3)" => Ok(Zfs(ZfsRaidLevel::RaidZ3)),
+ "btrfs (RAID0)" => Ok(Btrfs(BtrfsRaidLevel::Raid0)),
+ "btrfs (RAID1)" => Ok(Btrfs(BtrfsRaidLevel::Raid1)),
+ "btrfs (RAID10)" => Ok(Btrfs(BtrfsRaidLevel::Raid10)),
+ _ => Err(de::Error::custom("could not find file system: {de_fs}"))
+ }
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 29/30] common: skip target_hd when deserializing InstallConfig
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (27 preceding siblings ...)
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 28/30] common: add deserializer for FsType Aaron Lauterer
@ 2024-04-04 14:49 ` Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 30/30] add proxmox-chroot utility Aaron Lauterer
` (2 subsequent siblings)
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:49 UTC (permalink / raw)
To: pve-devel
as only the 'path' property is serialized -> deserialization is
problematic. The information would be present in the 'run-env-info-json',
but for now there is no need for it in any code that deserializes the
low-level config. Therefore we are currently skipping it on
deserialization
If we need it in the future, we need to think about how to handle the
deserialization.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
proxmox-installer-common/src/setup.rs | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs
index c580477..2454cbd 100644
--- a/proxmox-installer-common/src/setup.rs
+++ b/proxmox-installer-common/src/setup.rs
@@ -423,7 +423,12 @@ pub struct InstallConfig {
#[serde(
serialize_with = "serialize_disk_opt",
- skip_serializing_if = "Option::is_none"
+ skip_serializing_if = "Option::is_none",
+ // only the 'path' property is serialized -> deserialization is problematic
+ // The information would be present in the 'run-env-info-json', but for now there is no
+ // need for it in any code that deserializes the low-level config. Therefore we are
+ // currently skipping it on deserialization
+ skip_deserializing
)]
pub target_hd: Option<Disk>,
#[serde(skip_serializing_if = "BTreeMap::is_empty")]
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH installer v4 30/30] add proxmox-chroot utility
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (28 preceding siblings ...)
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 29/30] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
@ 2024-04-04 14:49 ` Aaron Lauterer
2024-04-05 12:38 ` [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Christoph Heiss
2024-04-05 14:25 ` [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields Aaron Lauterer
31 siblings, 0 replies; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-04 14:49 UTC (permalink / raw)
To: pve-devel
it is meant as a helper utility to prepare an installation for chroot
and clean up afterwards
It tries to determine the used FS from the previous installation, will
do what is necessary to mount/import the root FS to /target. It then
will set up all bind mounts.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Cargo.toml | 1 +
Makefile | 5 +-
proxmox-chroot/Cargo.toml | 16 ++
proxmox-chroot/src/main.rs | 356 +++++++++++++++++++++++++++++++++++++
4 files changed, 377 insertions(+), 1 deletion(-)
create mode 100644 proxmox-chroot/Cargo.toml
create mode 100644 proxmox-chroot/src/main.rs
diff --git a/Cargo.toml b/Cargo.toml
index b694d5b..b3afc7c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -2,6 +2,7 @@
members = [
"proxmox-auto-installer",
"proxmox-autoinst-helper",
+ "proxmox-chroot",
"proxmox-fetch-answer",
"proxmox-installer-common",
"proxmox-tui-installer",
diff --git a/Makefile b/Makefile
index e32d28f..d69dc6f 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,7 @@ INSTALLER_SOURCES=$(shell git ls-files) country.dat
PREFIX = /usr
BINDIR = $(PREFIX)/bin
USR_BIN := \
+ proxmox-chroot\
proxmox-tui-installer\
proxmox-autoinst-helper\
proxmox-fetch-answer\
@@ -54,6 +55,7 @@ $(BUILDDIR):
proxmox-auto-installer/ \
proxmox-autoinst-helper/ \
proxmox-fetch-answer/ \
+ proxmox-chroot \
proxmox-tui-installer/ \
proxmox-installer-common/ \
test/ \
@@ -127,7 +129,8 @@ cargo-build:
$(CARGO) build --package proxmox-tui-installer --bin proxmox-tui-installer \
--package proxmox-auto-installer --bin proxmox-auto-installer \
--package proxmox-fetch-answer --bin proxmox-fetch-answer \
- --package proxmox-autoinst-helper --bin proxmox-autoinst-helper $(CARGO_BUILD_ARGS)
+ --package proxmox-autoinst-helper --bin proxmox-autoinst-helper \
+ --package proxmox-chroot --bin proxmox-chroot $(CARGO_BUILD_ARGS)
%-banner.png: %-banner.svg
rsvg-convert -o $@ $<
diff --git a/proxmox-chroot/Cargo.toml b/proxmox-chroot/Cargo.toml
new file mode 100644
index 0000000..43b96ff
--- /dev/null
+++ b/proxmox-chroot/Cargo.toml
@@ -0,0 +1,16 @@
+[package]
+name = "proxmox-chroot"
+version = "0.1.0"
+edition = "2021"
+authors = [ "Aaron Lauterer <a.lauterer@proxmox.com>" ]
+license = "AGPL-3"
+exclude = [ "build", "debian" ]
+homepage = "https://www.proxmox.com"
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.0", features = ["derive"] }
+nix = "0.26.1"
+proxmox-installer-common = { path = "../proxmox-installer-common" }
+regex = "1.7"
+serde_json = "1.0"
diff --git a/proxmox-chroot/src/main.rs b/proxmox-chroot/src/main.rs
new file mode 100644
index 0000000..c1a4785
--- /dev/null
+++ b/proxmox-chroot/src/main.rs
@@ -0,0 +1,356 @@
+use std::{fs, io, path, process::Command};
+
+use anyhow::{bail, Result};
+use clap::{Args, Parser, Subcommand, ValueEnum};
+use nix::mount::{mount, umount, MsFlags};
+use proxmox_installer_common::{
+ options::FsType,
+ setup::{InstallConfig, SetupInfo},
+};
+use regex::Regex;
+
+const ANSWER_MP: &str = "answer";
+static BINDMOUNTS: [&str; 4] = ["dev", "proc", "run", "sys"];
+const TARGET_DIR: &str = "/target";
+const ZPOOL_NAME: &str = "rpool";
+
+/// Helper tool to prepare eveything to `chroot` into an installation
+#[derive(Parser, Debug)]
+#[command(author, version, about, long_about = None)]
+struct Cli {
+ #[command(subcommand)]
+ command: Commands,
+}
+
+#[derive(Subcommand, Debug)]
+enum Commands {
+ Prepare(CommandPrepare),
+ Cleanup(CommandCleanup),
+}
+
+/// Mount the root file system and bind mounts in preparation to chroot into the installation
+#[derive(Args, Debug)]
+struct CommandPrepare {
+ /// Filesystem used for the installation. Will try to automatically detect it after a
+ /// successful installation.
+ #[arg(short, long, value_enum)]
+ filesystem: Option<Filesystems>,
+
+ /// Numerical ID of `rpool` ZFS pool to import. Needed if multiple pools of name `rpool` are present.
+ #[arg(long)]
+ rpool_id: Option<u64>,
+
+ /// UUID of the BTRFS file system to mount. Needed if multiple BTRFS file systems are present.
+ #[arg(long)]
+ btrfs_uuid: Option<String>,
+}
+
+/// Unmount everything. Use once done with chroot.
+#[derive(Args, Debug)]
+struct CommandCleanup {
+ /// Filesystem used for the installation. Will try to automatically detect it by default.
+ #[arg(short, long, value_enum)]
+ filesystem: Option<Filesystems>,
+}
+
+#[derive(Copy, Clone, Debug, ValueEnum)]
+enum Filesystems {
+ Zfs,
+ Ext4,
+ Xfs,
+ Btrfs,
+}
+
+impl From<FsType> for Filesystems {
+ fn from(fs: FsType) -> Self {
+ match fs {
+ FsType::Xfs => Self::Xfs,
+ FsType::Ext4 => Self::Ext4,
+ FsType::Zfs(_) => Self::Zfs,
+ FsType::Btrfs(_) => Self::Btrfs,
+ }
+ }
+}
+
+fn main() {
+ let args = Cli::parse();
+ let res = match &args.command {
+ Commands::Prepare(args) => prepare(args),
+ Commands::Cleanup(args) => cleanup(args),
+ };
+ if let Err(err) = res {
+ eprintln!("{err}");
+ std::process::exit(1);
+ }
+}
+
+fn prepare(args: &CommandPrepare) -> Result<()> {
+ let fs = get_fs(args.filesystem)?;
+
+ fs::create_dir_all(TARGET_DIR)?;
+
+ match fs {
+ Filesystems::Zfs => mount_zpool(args.rpool_id)?,
+ Filesystems::Xfs => mount_fs()?,
+ Filesystems::Ext4 => mount_fs()?,
+ Filesystems::Btrfs => mount_btrfs(args.btrfs_uuid.clone())?,
+ }
+
+ if let Err(e) = bindmount() {
+ eprintln!("{e}")
+ }
+
+ println!("Done. You can now use 'chroot /target /bin/bash'!");
+ Ok(())
+}
+
+fn cleanup(args: &CommandCleanup) -> Result<()> {
+ let fs = get_fs(args.filesystem)?;
+
+ if let Err(e) = bind_umount() {
+ eprintln!("{e}")
+ }
+
+ match fs {
+ Filesystems::Zfs => umount_zpool(),
+ Filesystems::Xfs => umount_fs()?,
+ Filesystems::Ext4 => umount_fs()?,
+ _ => (),
+ }
+
+ println!("Chroot cleanup done. You can now reboot or leave the shell.");
+ Ok(())
+}
+
+fn get_fs(filesystem: Option<Filesystems>) -> Result<Filesystems> {
+ let fs = match filesystem {
+ None => {
+ let low_level_config = match get_low_level_config() {
+ Ok(c) => c,
+ Err(_) => bail!("Could not fetch config from previous installation. Please specify file system with -f."),
+ };
+ Filesystems::from(low_level_config.filesys)
+ }
+ Some(fs) => fs,
+ };
+
+ Ok(fs)
+}
+
+fn get_low_level_config() -> Result<InstallConfig> {
+ let file = fs::File::open("/tmp/low-level-config.json")?;
+ let reader = io::BufReader::new(file);
+ let config: InstallConfig = serde_json::from_reader(reader)?;
+ Ok(config)
+}
+
+fn get_iso_info() -> Result<SetupInfo> {
+ let file = fs::File::open("/run/proxmox-installer/iso-info.json")?;
+ let reader = io::BufReader::new(file);
+ let setup_info: SetupInfo = serde_json::from_reader(reader)?;
+ Ok(setup_info)
+}
+
+fn mount_zpool(pool_id: Option<u64>) -> Result<()> {
+ println!("importing ZFS pool to {TARGET_DIR}");
+ let mut import = Command::new("zpool");
+ import.arg("import").args(["-R", TARGET_DIR]);
+ match pool_id {
+ None => {
+ import.arg(ZPOOL_NAME);
+ }
+ Some(id) => {
+ import.arg(id.to_string());
+ }
+ }
+ match import.status() {
+ Ok(s) if !s.success() => bail!("Could not import ZFS pool. Abort!"),
+ _ => (),
+ }
+ println!("successfully imported ZFS pool to {TARGET_DIR}");
+ Ok(())
+}
+
+fn umount_zpool() {
+ match Command::new("zpool").arg("export").arg(ZPOOL_NAME).status() {
+ Ok(s) if !s.success() => println!("failure on exporting {ZPOOL_NAME}"),
+ _ => (),
+ }
+}
+
+fn mount_fs() -> Result<()> {
+ let iso_info = get_iso_info()?;
+ let product = iso_info.config.product;
+
+ println!("Activating VG '{product}'");
+ let res = Command::new("vgchange")
+ .arg("-ay")
+ .arg(product.to_string())
+ .output();
+ match res {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!(
+ "successfully activated VG '{product}': {}",
+ String::from_utf8(output.stdout)?
+ );
+ } else {
+ bail!(
+ "activation of VG '{product}' failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ match Command::new("mount")
+ .arg(format!("/dev/mapper/{product}-root"))
+ .arg("/target")
+ .output()
+ {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!("mounted root file system successfully");
+ } else {
+ bail!(
+ "mounting of root file system failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn umount_fs() -> Result<()> {
+ umount(TARGET_DIR)?;
+ Ok(())
+}
+
+fn mount_btrfs(btrfs_uuid: Option<String>) -> Result<()> {
+ let uuid = match btrfs_uuid {
+ Some(uuid) => uuid,
+ None => get_btrfs_uuid()?,
+ };
+
+ match Command::new("mount")
+ .arg("--uuid")
+ .arg(uuid)
+ .arg("/target")
+ .output()
+ {
+ Err(e) => bail!("{e}"),
+ Ok(output) => {
+ if output.status.success() {
+ println!("mounted BTRFS root file system successfully");
+ } else {
+ bail!(
+ "mounting of BTRFS root file system failed: {}",
+ String::from_utf8(output.stderr)?
+ )
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn get_btrfs_uuid() -> Result<String> {
+ let output = Command::new("btrfs")
+ .arg("filesystem")
+ .arg("show")
+ .output()?;
+ if !output.status.success() {
+ bail!(
+ "Error checking for BTRFS file systems: {}",
+ String::from_utf8(output.stderr)?
+ );
+ }
+ let out = String::from_utf8(output.stdout)?;
+ let mut uuids = Vec::new();
+
+ let re_uuid =
+ Regex::new(r"uuid: ([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$")?;
+ for line in out.lines() {
+ if let Some(cap) = re_uuid.captures(line) {
+ if let Some(uuid) = cap.get(1) {
+ uuids.push(uuid.as_str());
+ }
+ }
+ }
+ match uuids.len() {
+ 0 => bail!("Could not find any BTRFS UUID"),
+ i if i > 1 => {
+ let uuid_list = uuids
+ .iter()
+ .fold(String::new(), |acc, &arg| format!("{acc}\n{arg}"));
+ bail!("Found {i} UUIDs:{uuid_list}\nPlease specify the UUID to use with the --btrfs-uuid parameter")
+ }
+ _ => (),
+ }
+ Ok(uuids[0].into())
+}
+
+fn bindmount() -> Result<()> {
+ println!("Bind mounting");
+ // https://github.com/nix-rust/nix/blob/7badbee1e388618457ed0d725c1091359f253012/test/test_mount.rs#L19
+ // https://github.com/nix-rust/nix/blob/7badbee1e388618457ed0d725c1091359f253012/test/test_mount.rs#L146
+ const NONE: Option<&'static [u8]> = None;
+
+ let flags = MsFlags::MS_BIND;
+ for item in BINDMOUNTS {
+ let source = path::Path::new("/").join(item);
+ let target = path::Path::new(TARGET_DIR).join(item);
+
+ println!("Bindmount {} to {}", source.display(), target.display());
+ mount(Some(source.as_path()), target.as_path(), NONE, flags, NONE)?;
+ }
+
+ let answer_path = path::Path::new("/mnt").join(ANSWER_MP);
+ if answer_path.exists() {
+ let target = path::Path::new(TARGET_DIR).join("mnt").join(ANSWER_MP);
+
+ println!("Create dir {}", target.display());
+ fs::create_dir_all(&target)?;
+
+ println!(
+ "Bindmount {} to {}",
+ answer_path.display(),
+ target.display()
+ );
+ mount(
+ Some(answer_path.as_path()),
+ target.as_path(),
+ NONE,
+ flags,
+ NONE,
+ )?;
+ }
+ Ok(())
+}
+
+fn bind_umount() -> Result<()> {
+ for item in BINDMOUNTS {
+ let target = path::Path::new(TARGET_DIR).join(item);
+ println!("Unmounting {}", target.display());
+ if let Err(e) = umount(target.as_path()) {
+ eprintln!("{e}");
+ }
+ }
+
+ let answer_target = path::Path::new(TARGET_DIR).join("mnt").join(ANSWER_MP);
+ if answer_target.exists() {
+ println!("Unmounting and removing answer mountpoint");
+ if let Err(e) = umount(answer_target.as_os_str()) {
+ eprintln!("{e}");
+ }
+ if let Err(e) = fs::remove_dir(answer_target) {
+ eprintln!("{e}");
+ }
+ }
+
+ Ok(())
+}
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (29 preceding siblings ...)
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 30/30] add proxmox-chroot utility Aaron Lauterer
@ 2024-04-05 12:38 ` Christoph Heiss
2024-04-05 14:25 ` [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields Aaron Lauterer
31 siblings, 0 replies; 34+ messages in thread
From: Christoph Heiss @ 2024-04-05 12:38 UTC (permalink / raw)
To: Aaron Lauterer; +Cc: pve-devel
I've tested mostly the same things as for v3 [0], to confirm nothing
broke since that:
- Using a few different values for `global` options
- Install on ext4, xfs, Btrfs RAID1 and ZFS RAID10
(with different values in multiple runs)
- Using DHCP and static IP
- Fetching answer from a partition
- Fetching answer from a HTTP source, getting the URL through DHCP or
DNS
- Trying out the `proxmox-autoinst-helper` tool for assembling udev
rules and validating files.
- Using the `post_command` to create some files in the newly installed
system.
- Tested with PVE, PMG and PBS, each w/ BIOS & UEFI (latter also w/ SB)
One small thing I noticed: unknown/undefined options in the answer file
are currently silently ignored - in the installer as well as by
`proxmox-autoinst-helper validate-answer`.
Something to implement in the future though definitely, but for now IMHO
a rather mundane issue. Really just noting it here for reference.
I can also confirm now that a small bug I found in [0] is now fixed,
such that LVM configurations only allows a single disk now.
The other things from [0] (and more) were also talked over again with
Aaron directly, off-list.
Also quickly skimmed over the actual changes again, looks fine overall.
At least nothing to really note of; that would impact functionality and
aren't some low-hanging fruit for the future (as e.g. noted above).
So please consider this whole series:
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
[0] https://lists.proxmox.com/pipermail/pve-devel/2024-April/062485.html
On Thu, Apr 04, 2024 at 04:48:32PM +0200, Aaron Lauterer wrote:
> This patch series adds the possibility to do an automated / unattended
> installation of Proxmox VE.
>
> The overall idea is that we will have a dedicated ISO for the unattended
> installation. It should be configured in such a way that it will start
> the installation without any user interaction. Therefore, the GRUB
> config should automatically start it (after a timeout).
>
> The information for the installer that is usually gathered interactively
> from the user is provided via an `answer.toml` file.
>
> The answer file allows to select disks and the network card via filters.
>
> The installer also allows to run custom commands pre and post
> installation. This should give users plenty of possibilities to either
> further customize/prepare the installation or integrate it into a larger
> automated installation setup.
> For example, one could issue HTTP requests to signal the status and
> progress of the installation.
>
> When the installer is called with 'proxauto' in the kernel cmdline, the
> 'proxmox-fetch-answer' binary is called. It tries to find the answer
> file and once found, will start the 'proxmox-auto-installer' binary and
> pass the contents to it via stdin.
>
> The auto-installer then parses the answer file and determines what
> parameters need to be passed to the low-level installer. For example,
> which disks and NIC to use, network IP settings and so forth.
>
> The current status reporting of the actual installation is kept rather
> simple.
>
> Both binaries log into the tmp directory.
>
> There is a third binary, the 'proxmox-autoinst-helper'. It provides a
> few subcommands, from the help:
> answer Validate if an answer file is formatted correctly
> device-match Test which devices the given filter matches against
> device-info Show device information that can be used for filters
> identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file
>
> The fetch-answer binary is trying to get an answer file. It does so by
> first searching for a partition/FS labeled `proxmoxinst`, or all upper
> case, and an `answer.toml` in there. This could be provided by another
> USB flash drive.
> If that is not successful, the next step is to send an HTTP POST request
> to a URL to get the TOML contents in return. A POST request was chosen
> because we also send information to identify the host in JSON format.
>
> The question then is, where to get that URL from. Right now, there are
> two options implemented. The first is looking for a custom DHCP option
> and the second is querying for a TXT record in the `proxmoxinst`
> subdomain of the search domain.
>
> It is possible to provide a SHA256 fingerprint of the SSL cert used by
> the answer server. The safest option is to place a
> `cert_fingerprint.txt` file in the same `proxmoxinst` partition as where
> you alternatively would place the `answer.toml`.
> If that is not found, then it can be provided by a second custom DHCP
> option or placed as TXT record in the subdomain `proxmoxinst-fp`.
>
> This patch series now also separates the 3 binaries into their own
> crate. The 'proxmox-fetch-answer' to keep the OpenSSL dependency as
> localized as possible, and the 'proxmox-autoinst-helper' to make it easy
> to compile just that binary.
>
> The new `proxmox-chroot` utility helps to prepare everything to chroot
> into a fresh installation and clean it up once done.
> This will be useful in the post commands when further customizing the
> installation.
>
>
> Other plans / ideas for the future:
>
> * add option to define remote SSH access (password and,or public key).
> This could make remote debugging in case of problems easier
>
>
> Regarding the patch series itself:
> 01-03 are needed to move some code into the common crate and
> make structs/functions already in the common crate accessible.
>
> I did split up the individual parts of the auto installer into their own
> patches as much as possible, and (hopefully) in the order they depend on
> each other.
>
> Patches after the `unconfigured` one (16), switch the pattern matching
> to the glob crate, add the helper tool and the fetching via HTTP.
>
> Patch 26 factors our the binaries into their own crates.
>
> Patches 27-30 are for the 'proxmox-chroot' utility and preparations for
> it to work.
>
> Areas that can be improved/extended:
> * Testing possibility integrated in the Makefile
>
> I did test it with all 3 installers, PVE, PMG and PBS and it worked.
>
> WIP: Documentation. A first draft is available in the inernal wiki, as
> we will most likely keep it in wiki format since it applies for all 3
> products, if we provide ISOs for it.
>
> since v3:
>
> Tested-by: Christoph Heiss <c.heiss@proxmox.com>
>
> Changes since V3:
> * implement most suggested code changes. Thx @Christoph for reviewing it
> * reordered patches a little bit. While testing individual changes I
> realized that some patches needed reordering and rebasing
> * improved error handling of pre- and post-commands. Errors will now be
> logged & printed.
>
> Changes since v2:
> * don't use 'dmidecode' but check in the source locations directly for
> identifiers
> * fixed makefile to get builds working every time
> * reworked filesystem and raid level layout in the answer file
> definition
> * factor out binaries into their own crates
> * add 'proxmox-chroot' helper tool
>
> Changes since v1:
> * incorporated feedback (thx @cheiss)
> * the proxmox-installer-filter tool got renamed and extended to the
> proxmox-autoinst-helper
> * added HTTP fetch functionality
> * moved the fetch_plugins/utils.rs into its own subdirectory as it grew
> quite a bit
> * added builddeps
>
> Aaron Lauterer (30):
> tui: common: move InstallConfig struct to common crate
> common: make InstallZfsOption members public
> common: tui: use BTreeMap for predictable ordering
> common: utils: add deserializer for CidrAddress
> common: options: add Deserialize trait
> low-level: add dump-udev command
> add auto-installer crate
> auto-installer: add dependencies
> auto-installer: add answer file definition
> auto-installer: add struct to hold udev info
> auto-installer: add utils
> auto-installer: add simple logging
> auto-installer: add tests for answer file parsing
> auto-installer: add auto-installer binary
> auto-installer: add fetch answer binary
> unconfigured: add proxauto as option to start auto installer
> auto-installer: use glob crate for pattern matching
> auto-installer: utils: make get_udev_index functions public
> auto-installer: add proxmox-autoinst-helper tool
> common: add Display trait to ProxmoxProduct
> auto-installer: fetch: add gathering of system identifiers and
> restructure code
> auto-installer: helper: add subcommand to view indentifiers
> auto-installer: fetch: add http post utility module
> auto-installer: fetch: add http plugin to fetch answer
> control: update build depends for auto installer
> auto installer: factor out fetch-answer and autoinst-helper
> low-level: write low level config to /tmp
> common: add deserializer for FsType
> common: skip target_hd when deserializing InstallConfig
> add proxmox-chroot utility
>
> Cargo.toml | 4 +
> Makefile | 19 +-
> Proxmox/Makefile | 1 +
> Proxmox/Sys/Udev.pm | 54 +++
> debian/control | 10 +
> proxmox-auto-installer/Cargo.toml | 20 +
> proxmox-auto-installer/src/answer.rs | 249 ++++++++++
> .../src/bin/proxmox-auto-installer.rs | 195 ++++++++
> proxmox-auto-installer/src/lib.rs | 5 +
> proxmox-auto-installer/src/log.rs | 38 ++
> proxmox-auto-installer/src/sysinfo.rs | 81 ++++
> proxmox-auto-installer/src/udevinfo.rs | 9 +
> proxmox-auto-installer/src/utils.rs | 437 ++++++++++++++++++
> proxmox-auto-installer/tests/parse-answer.rs | 106 +++++
> .../tests/resources/iso-info.json | 1 +
> .../tests/resources/locales.json | 1 +
> .../resources/parse_answer/disk_match.json | 29 ++
> .../resources/parse_answer/disk_match.toml | 17 +
> .../parse_answer/disk_match_all.json | 26 ++
> .../parse_answer/disk_match_all.toml | 17 +
> .../parse_answer/disk_match_any.json | 33 ++
> .../parse_answer/disk_match_any.toml | 17 +
> .../tests/resources/parse_answer/minimal.json | 17 +
> .../tests/resources/parse_answer/minimal.toml | 14 +
> .../resources/parse_answer/nic_matching.json | 17 +
> .../resources/parse_answer/nic_matching.toml | 19 +
> .../tests/resources/parse_answer/readme | 4 +
> .../resources/parse_answer/specific_nic.json | 17 +
> .../resources/parse_answer/specific_nic.toml | 19 +
> .../tests/resources/parse_answer/zfs.json | 27 ++
> .../tests/resources/parse_answer/zfs.toml | 20 +
> .../tests/resources/run-env-info.json | 1 +
> .../tests/resources/run-env-udev.json | 1 +
> proxmox-autoinst-helper/Cargo.toml | 21 +
> proxmox-autoinst-helper/src/main.rs | 337 ++++++++++++++
> proxmox-chroot/Cargo.toml | 16 +
> proxmox-chroot/src/main.rs | 356 ++++++++++++++
> proxmox-fetch-answer/Cargo.toml | 22 +
> .../src/fetch_plugins/http.rs | 191 ++++++++
> proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 +
> .../src/fetch_plugins/partition.rs | 32 ++
> .../src/fetch_plugins/utils/mod.rs | 85 ++++
> .../src/fetch_plugins/utils/post.rs | 94 ++++
> proxmox-fetch-answer/src/main.rs | 76 +++
> proxmox-installer-common/Cargo.toml | 1 +
> proxmox-installer-common/src/options.rs | 21 +-
> proxmox-installer-common/src/setup.rs | 141 +++++-
> proxmox-installer-common/src/utils.rs | 11 +
> proxmox-low-level-installer | 14 +
> proxmox-tui-installer/src/options.rs | 4 +-
> proxmox-tui-installer/src/setup.rs | 100 +---
> .../src/views/install_progress.rs | 4 +-
> unconfigured.sh | 17 +
> 53 files changed, 2953 insertions(+), 118 deletions(-)
> create mode 100644 Proxmox/Sys/Udev.pm
> create mode 100644 proxmox-auto-installer/Cargo.toml
> create mode 100644 proxmox-auto-installer/src/answer.rs
> create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
> create mode 100644 proxmox-auto-installer/src/lib.rs
> create mode 100644 proxmox-auto-installer/src/log.rs
> create mode 100644 proxmox-auto-installer/src/sysinfo.rs
> create mode 100644 proxmox-auto-installer/src/udevinfo.rs
> create mode 100644 proxmox-auto-installer/src/utils.rs
> create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
> create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
> create mode 100644 proxmox-auto-installer/tests/resources/locales.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
> create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
> create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
> create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
> create mode 100644 proxmox-autoinst-helper/Cargo.toml
> create mode 100644 proxmox-autoinst-helper/src/main.rs
> create mode 100644 proxmox-chroot/Cargo.toml
> create mode 100644 proxmox-chroot/src/main.rs
> create mode 100644 proxmox-fetch-answer/Cargo.toml
> create mode 100644 proxmox-fetch-answer/src/fetch_plugins/http.rs
> create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
> create mode 100644 proxmox-fetch-answer/src/fetch_plugins/partition.rs
> create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
> create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
> create mode 100644 proxmox-fetch-answer/src/main.rs
>
> --
> 2.39.2
>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
` (30 preceding siblings ...)
2024-04-05 12:38 ` [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Christoph Heiss
@ 2024-04-05 14:25 ` Aaron Lauterer
2024-04-09 9:20 ` Christoph Heiss
31 siblings, 1 reply; 34+ messages in thread
From: Aaron Lauterer @ 2024-04-05 14:25 UTC (permalink / raw)
To: pve-devel
This way, serde will throw errors if fields are not known.
This can help to reduce frustration if one might think to have set an
option, but for example a small type has happened.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Since Christoph mentioned it I tried to implement it. Tested quickly
with the proxmox-autoinst-helper tool.
proxmox-auto-installer/src/answer.rs | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
index 94cebb3..57c2602 100644
--- a/proxmox-auto-installer/src/answer.rs
+++ b/proxmox-auto-installer/src/answer.rs
@@ -10,7 +10,7 @@ use std::{collections::BTreeMap, net::IpAddr};
/// storing them in a HashMap
#[derive(Clone, Deserialize, Debug)]
-#[serde(rename_all = "kebab-case")]
+#[serde(rename_all = "kebab-case", deny_unknown_fields)]
pub struct Answer {
pub global: Global,
pub network: Network,
@@ -19,6 +19,7 @@ pub struct Answer {
}
#[derive(Clone, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct Global {
pub country: String,
pub fqdn: Fqdn,
@@ -33,6 +34,7 @@ pub struct Global {
}
#[derive(Clone, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
struct NetworkInAnswer {
#[serde(default)]
pub use_dhcp: bool,
@@ -43,7 +45,7 @@ struct NetworkInAnswer {
}
#[derive(Clone, Deserialize, Debug)]
-#[serde(try_from = "NetworkInAnswer")]
+#[serde(try_from = "NetworkInAnswer", deny_unknown_fields)]
pub struct Network {
pub network_settings: NetworkSettings,
}
@@ -97,6 +99,7 @@ pub struct NetworkManual {
}
#[derive(Clone, Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
pub struct DiskSetup {
pub filesystem: Filesystem,
#[serde(default)]
@@ -109,7 +112,7 @@ pub struct DiskSetup {
}
#[derive(Clone, Debug, Deserialize)]
-#[serde(try_from = "DiskSetup")]
+#[serde(try_from = "DiskSetup", deny_unknown_fields)]
pub struct Disks {
pub fs_type: FsType,
pub disk_selection: DiskSelection,
@@ -207,14 +210,14 @@ pub enum DiskSelection {
Filter(BTreeMap<String, String>),
}
#[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)]
-#[serde(rename_all = "lowercase")]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
pub enum FilterMatch {
Any,
All,
}
#[derive(Clone, Deserialize, Serialize, Debug, PartialEq)]
-#[serde(rename_all = "lowercase")]
+#[serde(rename_all = "lowercase", deny_unknown_fields)]
pub enum Filesystem {
Ext4,
Xfs,
@@ -223,6 +226,7 @@ pub enum Filesystem {
}
#[derive(Clone, Copy, Default, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct ZfsOptions {
pub raid: Option<ZfsRaidLevel>,
pub ashift: Option<usize>,
@@ -234,6 +238,7 @@ pub struct ZfsOptions {
}
#[derive(Clone, Copy, Default, Deserialize, Serialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct LvmOptions {
pub hdsize: Option<f64>,
pub swapsize: Option<f64>,
@@ -243,6 +248,7 @@ pub struct LvmOptions {
}
#[derive(Clone, Copy, Default, Deserialize, Debug)]
+#[serde(deny_unknown_fields)]
pub struct BtrfsOptions {
pub hdsize: Option<f64>,
pub raid: Option<BtrfsRaidLevel>,
--
2.39.2
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields
2024-04-05 14:25 ` [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields Aaron Lauterer
@ 2024-04-09 9:20 ` Christoph Heiss
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Heiss @ 2024-04-09 9:20 UTC (permalink / raw)
To: Aaron Lauterer; +Cc: Proxmox VE development discussion
LGTM, does exactly what is says on the tin.
Tested it using both the `proxmox-autoinst-helper validate-answer` tool
and trying to boot the auto-installer itself with a bogus answer file.
So please consider this also:
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
On Fri, Apr 05, 2024 at 04:25:07PM +0200, Aaron Lauterer wrote:
> This way, serde will throw errors if fields are not known.
>
> This can help to reduce frustration if one might think to have set an
> option, but for example a small type has happened.
Yeah, that's kinda how I discovered that, wondering why a certain
option did not get applied :^)
>
> Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
> ---
> Since Christoph mentioned it I tried to implement it. Tested quickly
> with the proxmox-autoinst-helper tool.
>
> proxmox-auto-installer/src/answer.rs | 16 +++++++++++-----
> 1 file changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/proxmox-auto-installer/src/answer.rs b/proxmox-auto-installer/src/answer.rs
> index 94cebb3..57c2602 100644
> --- a/proxmox-auto-installer/src/answer.rs
> +++ b/proxmox-auto-installer/src/answer.rs
> @@ -10,7 +10,7 @@ use std::{collections::BTreeMap, net::IpAddr};
> /// storing them in a HashMap
>
> #[derive(Clone, Deserialize, Debug)]
> -#[serde(rename_all = "kebab-case")]
> +#[serde(rename_all = "kebab-case", deny_unknown_fields)]
> pub struct Answer {
> pub global: Global,
> pub network: Network,
> @@ -19,6 +19,7 @@ pub struct Answer {
> }
>
> #[derive(Clone, Deserialize, Debug)]
> +#[serde(deny_unknown_fields)]
> pub struct Global {
> pub country: String,
> pub fqdn: Fqdn,
> @@ -33,6 +34,7 @@ pub struct Global {
> }
>
> #[derive(Clone, Deserialize, Debug)]
> +#[serde(deny_unknown_fields)]
> struct NetworkInAnswer {
> #[serde(default)]
> pub use_dhcp: bool,
> @@ -43,7 +45,7 @@ struct NetworkInAnswer {
> }
>
> #[derive(Clone, Deserialize, Debug)]
> -#[serde(try_from = "NetworkInAnswer")]
> +#[serde(try_from = "NetworkInAnswer", deny_unknown_fields)]
> pub struct Network {
> pub network_settings: NetworkSettings,
> }
> @@ -97,6 +99,7 @@ pub struct NetworkManual {
> }
>
> #[derive(Clone, Debug, Deserialize)]
> +#[serde(deny_unknown_fields)]
> pub struct DiskSetup {
> pub filesystem: Filesystem,
> #[serde(default)]
> @@ -109,7 +112,7 @@ pub struct DiskSetup {
> }
>
> #[derive(Clone, Debug, Deserialize)]
> -#[serde(try_from = "DiskSetup")]
> +#[serde(try_from = "DiskSetup", deny_unknown_fields)]
> pub struct Disks {
> pub fs_type: FsType,
> pub disk_selection: DiskSelection,
> @@ -207,14 +210,14 @@ pub enum DiskSelection {
> Filter(BTreeMap<String, String>),
> }
> #[derive(Clone, Deserialize, Debug, PartialEq, ValueEnum)]
> -#[serde(rename_all = "lowercase")]
> +#[serde(rename_all = "lowercase", deny_unknown_fields)]
> pub enum FilterMatch {
> Any,
> All,
> }
>
> #[derive(Clone, Deserialize, Serialize, Debug, PartialEq)]
> -#[serde(rename_all = "lowercase")]
> +#[serde(rename_all = "lowercase", deny_unknown_fields)]
> pub enum Filesystem {
> Ext4,
> Xfs,
> @@ -223,6 +226,7 @@ pub enum Filesystem {
> }
>
> #[derive(Clone, Copy, Default, Deserialize, Debug)]
> +#[serde(deny_unknown_fields)]
> pub struct ZfsOptions {
> pub raid: Option<ZfsRaidLevel>,
> pub ashift: Option<usize>,
> @@ -234,6 +238,7 @@ pub struct ZfsOptions {
> }
>
> #[derive(Clone, Copy, Default, Deserialize, Serialize, Debug)]
> +#[serde(deny_unknown_fields)]
> pub struct LvmOptions {
> pub hdsize: Option<f64>,
> pub swapsize: Option<f64>,
> @@ -243,6 +248,7 @@ pub struct LvmOptions {
> }
>
> #[derive(Clone, Copy, Default, Deserialize, Debug)]
> +#[serde(deny_unknown_fields)]
> pub struct BtrfsOptions {
> pub hdsize: Option<f64>,
> pub raid: Option<BtrfsRaidLevel>,
> --
> 2.39.2
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2024-04-09 9:20 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-04 14:48 [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 01/30] tui: common: move InstallConfig struct to common crate Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 02/30] common: make InstallZfsOption members public Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 03/30] common: tui: use BTreeMap for predictable ordering Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 04/30] common: utils: add deserializer for CidrAddress Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 05/30] common: options: add Deserialize trait Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 06/30] low-level: add dump-udev command Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 07/30] add auto-installer crate Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 08/30] auto-installer: add dependencies Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 09/30] auto-installer: add answer file definition Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 10/30] auto-installer: add struct to hold udev info Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 11/30] auto-installer: add utils Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 12/30] auto-installer: add simple logging Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 13/30] auto-installer: add tests for answer file parsing Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 14/30] auto-installer: add auto-installer binary Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 15/30] auto-installer: add fetch answer binary Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 16/30] unconfigured: add proxauto as option to start auto installer Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 17/30] auto-installer: use glob crate for pattern matching Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 18/30] auto-installer: utils: make get_udev_index functions public Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 19/30] auto-installer: add proxmox-autoinst-helper tool Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 20/30] common: add Display trait to ProxmoxProduct Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 21/30] auto-installer: fetch: add gathering of system identifiers and restructure code Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 22/30] auto-installer: helper: add subcommand to view indentifiers Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 23/30] auto-installer: fetch: add http post utility module Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 24/30] auto-installer: fetch: add http plugin to fetch answer Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 25/30] control: update build depends for auto installer Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 26/30] auto installer: factor out fetch-answer and autoinst-helper Aaron Lauterer
2024-04-04 14:48 ` [pve-devel] [PATCH installer v4 27/30] low-level: write low level config to /tmp Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 28/30] common: add deserializer for FsType Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 29/30] common: skip target_hd when deserializing InstallConfig Aaron Lauterer
2024-04-04 14:49 ` [pve-devel] [PATCH installer v4 30/30] add proxmox-chroot utility Aaron Lauterer
2024-04-05 12:38 ` [pve-devel] [PATCH installer v4 00/30] add automated/unattended installation Christoph Heiss
2024-04-05 14:25 ` [pve-devel] [PATCH v4 installer 31/30 follow-up] auto-installer: answer: deny unknown fields Aaron Lauterer
2024-04-09 9:20 ` Christoph Heiss
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox