* [pve-devel] [PATCH installer v2 1/4] tree-wide: run rustfmt, fix clippy warnings
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
@ 2024-05-13 9:49 ` Christoph Heiss
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 2/4] assistant: drop unused `log` dependency Christoph Heiss
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Christoph Heiss @ 2024-05-13 9:49 UTC (permalink / raw)
To: pve-devel
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
* no changes
proxmox-auto-installer/tests/parse-answer.rs | 14 +++++++-------
.../src/fetch_plugins/partition.rs | 10 +++++-----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs
index 4014b6d..e77a769 100644
--- a/proxmox-auto-installer/tests/parse-answer.rs
+++ b/proxmox-auto-installer/tests/parse-answer.rs
@@ -1,4 +1,4 @@
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
use serde_json::Value;
use std::fs;
@@ -24,9 +24,9 @@ fn get_answer(path: PathBuf) -> Result<Answer, String> {
Ok(answer)
}
-fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
+fn setup_test_basic(path: &Path) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
let installer_info: SetupInfo = {
- let mut path = path.clone();
+ let mut path = path.to_path_buf();
path.push("iso-info.json");
read_json(&path)
@@ -35,7 +35,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
};
let locale_info = {
- let mut path = path.clone();
+ let mut path = path.to_path_buf();
path.push("locales.json");
read_json(&path)
@@ -44,7 +44,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
};
let mut runtime_info: RuntimeInfo = {
- let mut path = path.clone();
+ let mut path = path.to_path_buf();
path.push("run-env-info.json");
read_json(&path)
@@ -53,7 +53,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
};
let udev_info: UdevInfo = {
- let mut path = path.clone();
+ let mut path = path.to_path_buf();
path.push("run-env-udev.json");
read_json(&path)
@@ -71,7 +71,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
fn test_parse_answers() {
let path = get_test_resource_path().unwrap();
let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path);
- let mut tests_path = path.clone();
+ let mut tests_path = path;
tests_path.push("parse_answer");
let test_dir = fs::read_dir(tests_path.clone()).unwrap();
diff --git a/proxmox-fetch-answer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
index 0479c8f..7213493 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -31,7 +31,7 @@ impl FetchFromPartition {
}
fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
- let path = Path::new(search_path).join(&file_name);
+ let path = Path::new(search_path).join(file_name);
info!("Testing partition search path {path:?}");
match path.try_exists() {
Ok(true) => Some(path),
@@ -51,14 +51,14 @@ fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
fn scan_partlabels(partlabel: &str, search_path: &str) -> Result<PathBuf> {
let partlabel_upper_case = partlabel.to_uppercase();
if let Some(path) = path_exists_logged(&partlabel_upper_case, search_path) {
- info!("Found partition with label '{partlabel_upper_case}'");
- return Ok(path);
+ info!("Found partition with label '{partlabel_upper_case}'");
+ return Ok(path);
}
let partlabel_lower_case = partlabel.to_lowercase();
if let Some(path) = path_exists_logged(&partlabel_lower_case, search_path) {
- info!("Found partition with label '{partlabel_lower_case}'");
- return Ok(path);
+ info!("Found partition with label '{partlabel_lower_case}'");
+ return Ok(path);
}
bail!("Could not detect upper or lower case labels for '{partlabel}'");
--
2.44.0
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread* [pve-devel] [PATCH installer v2 2/4] assistant: drop unused `log` dependency
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 1/4] tree-wide: run rustfmt, fix clippy warnings Christoph Heiss
@ 2024-05-13 9:49 ` Christoph Heiss
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 3/4] assistant: pre-compile ignored block device patterns Christoph Heiss
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Christoph Heiss @ 2024-05-13 9:49 UTC (permalink / raw)
To: pve-devel
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
* no changes
proxmox-auto-install-assistant/Cargo.toml | 1 -
1 file changed, 1 deletion(-)
diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index eaca7f8..0286c80 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -14,7 +14,6 @@ homepage = "https://www.proxmox.com"
anyhow = "1.0"
clap = { version = "4.0", features = ["derive"] }
glob = "0.3"
-log = "0.4.20"
proxmox-auto-installer = { path = "../proxmox-auto-installer" }
regex = "1.7"
serde = { version = "1.0", features = ["derive"] }
--
2.44.0
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread* [pve-devel] [PATCH installer v2 3/4] assistant: pre-compile ignored block device patterns
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 1/4] tree-wide: run rustfmt, fix clippy warnings Christoph Heiss
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 2/4] assistant: drop unused `log` dependency Christoph Heiss
@ 2024-05-13 9:49 ` Christoph Heiss
2024-11-12 19:55 ` [pve-devel] applied: " Thomas Lamprecht
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 4/4] assistant: avoid regex for simple prefix matching Christoph Heiss
` (2 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Christoph Heiss @ 2024-05-13 9:49 UTC (permalink / raw)
To: pve-devel
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
* no changes
proxmox-auto-install-assistant/src/main.rs | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 1447175..790dbc7 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -430,13 +430,13 @@ fn get_iso_uuid(iso: &PathBuf) -> Result<String> {
}
fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
- let unwantend_block_devs = vec![
- "ram[0-9]*",
- "loop[0-9]*",
- "md[0-9]*",
- "dm-*",
- "fd[0-9]*",
- "sr[0-9]*",
+ let unwanted_block_devs = [
+ Pattern::new("ram[0-9]*")?,
+ Pattern::new("loop[0-9]*")?,
+ Pattern::new("md[0-9]*")?,
+ Pattern::new("dm-*")?,
+ Pattern::new("fd[0-9]*")?,
+ Pattern::new("sr[0-9]*")?,
];
// compile Regex here once and not inside the loop
@@ -453,8 +453,8 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let entry = entry.unwrap();
let filename = entry.file_name().into_string().unwrap();
- for p in &unwantend_block_devs {
- if Pattern::new(p)?.matches(&filename) {
+ for p in &unwanted_block_devs {
+ if p.matches(&filename) {
continue 'outer;
}
}
--
2.44.0
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread* [pve-devel] [PATCH installer v2 4/4] assistant: avoid regex for simple prefix matching
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
` (2 preceding siblings ...)
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 3/4] assistant: pre-compile ignored block device patterns Christoph Heiss
@ 2024-05-13 9:49 ` Christoph Heiss
2024-05-13 11:15 ` [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Aaron Lauterer
2024-06-17 7:11 ` Christoph Heiss
5 siblings, 0 replies; 8+ messages in thread
From: Christoph Heiss @ 2024-05-13 9:49 UTC (permalink / raw)
To: pve-devel
udev properties are very easy to parse and can be done by doing a
line-based scan and matching the prefix, splitting once for properties.
Avoids the use of regexes and signicantly reduces binary size by about
-38%(!).
Tested by comparing the output of `proxmox-auto-install-assistant
device-info`, running it before and after the changes.
Stripped binary size for release builds:
before: 3103104 bytes ~ 2.96MiB
after: 1935744 bytes ~ 1.85MiB
text data bss dec hex filename
2906765 187144 537 3094446 2f37ae proxmox-auto-install-assistant-before
1871090 55736 497 1927323 1d689b proxmox-auto-install-assistant-after
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
* rebased on latest master
proxmox-auto-install-assistant/Cargo.toml | 1 -
proxmox-auto-install-assistant/src/main.rs | 57 +++++++++-------------
2 files changed, 24 insertions(+), 34 deletions(-)
diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index 0286c80..766b445 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -15,7 +15,6 @@ anyhow = "1.0"
clap = { version = "4.0", features = ["derive"] }
glob = "0.3"
proxmox-auto-installer = { path = "../proxmox-auto-installer" }
-regex = "1.7"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.7"
diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 790dbc7..7c0b0c6 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -1,7 +1,6 @@
use anyhow::{bail, format_err, Result};
use clap::{Args, Parser, Subcommand, ValueEnum};
use glob::Pattern;
-use regex::Regex;
use serde::Serialize;
use std::{
collections::BTreeMap,
@@ -439,13 +438,9 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
Pattern::new("sr[0-9]*")?,
];
- // compile Regex here once and not inside the loop
- let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?;
- let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?;
- let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?;
-
- let re_name = Regex::new(r"(?m)^N: (.*)$")?;
- let re_props = Regex::new(r"(?m)^E: ([^=]+)=(.*)$")?;
+ const PROP_DEVTYP_PREFIX: &str = "E: DEVTYPE=";
+ const PROP_CDROM: &str = "E: ID_CDROM";
+ const PROP_ISO9660_FS: &str = "E: ID_FS_TYPE=iso9660";
let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
@@ -467,30 +462,27 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
}
};
- if !re_disk.is_match(&output) {
- continue 'outer;
- };
- if re_cdrom.is_match(&output) {
- continue 'outer;
- };
- if re_iso9660.is_match(&output) {
- continue 'outer;
- };
-
let mut name = filename;
- if let Some(cap) = re_name.captures(&output) {
- if let Some(res) = cap.get(1) {
- name = String::from(res.as_str());
+ let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+ for line in output.lines() {
+ if let Some(prop) = line.strip_prefix(PROP_DEVTYP_PREFIX) {
+ if prop != "disk" {
+ continue 'outer;
+ }
}
- }
- let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+ if line.starts_with(PROP_CDROM) || line.starts_with(PROP_ISO9660_FS) {
+ continue 'outer;
+ }
- for line in output.lines() {
- if let Some(caps) = re_props.captures(line) {
- let key = String::from(caps.get(1).unwrap().as_str());
- let value = String::from(caps.get(2).unwrap().as_str());
- udev_props.insert(key, value);
+ if let Some(prop) = line.strip_prefix("N: ") {
+ name = prop.to_owned();
+ };
+
+ if let Some(prop) = line.strip_prefix("E: ") {
+ if let Some((key, val)) = prop.split_once('=') {
+ udev_props.insert(key.to_owned(), val.to_owned());
+ }
}
}
@@ -500,7 +492,6 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
}
fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
- let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
let links = get_nic_list()?;
@@ -518,10 +509,10 @@ fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
for line in output.lines() {
- if let Some(caps) = re_props.captures(line) {
- let key = String::from(caps.get(1).unwrap().as_str());
- let value = String::from(caps.get(2).unwrap().as_str());
- udev_props.insert(key, value);
+ if let Some(prop) = line.strip_prefix("E: ") {
+ if let Some((key, val)) = prop.split_once('=') {
+ udev_props.insert(key.to_owned(), val.to_owned());
+ }
}
}
--
2.44.0
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
` (3 preceding siblings ...)
2024-05-13 9:49 ` [pve-devel] [PATCH installer v2 4/4] assistant: avoid regex for simple prefix matching Christoph Heiss
@ 2024-05-13 11:15 ` Aaron Lauterer
2024-06-17 7:11 ` Christoph Heiss
5 siblings, 0 replies; 8+ messages in thread
From: Aaron Lauterer @ 2024-05-13 11:15 UTC (permalink / raw)
To: Proxmox VE development discussion, Christoph Heiss
did some tests on my local machine with the `device-info` and
`device-match` subcommands to list and match against identifiers of
disks and my NIC.
Thanks for taking the time to improve this code.
consider this series:
Reviewed-By: Aaron Lauterer <a.lauterer@proxmox.com>
Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>
On 2024-05-13 11:49, Christoph Heiss wrote:
> The proxmox-auto-install-assistant uses
> - glob patterns for disk matching, which can be pre-compiled for
> efficiency
> - regexes for udev property matching, which can be simplified by some
> simple prefix matching & splitting on =
>
> The latter also significantly reduces binary size due to the removing
> the regex dependency, for details see patch #4.
>
> Overall no functional changes in this series.
>
> v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063802.html
>
> Changes v1 -> v2:
> * rebased on latest master
>
> Christoph Heiss (4):
> tree-wide: run rustfmt, fix clippy warnings
> assistant: drop unused `log` dependency
> assistant: pre-compile ignored block device patterns
> assistant: avoid regex for simple prefix matching
>
> proxmox-auto-install-assistant/Cargo.toml | 2 -
> proxmox-auto-install-assistant/src/main.rs | 75 ++++++++-----------
> proxmox-auto-installer/tests/parse-answer.rs | 14 ++--
> .../src/fetch_plugins/partition.rs | 10 +--
> 4 files changed, 45 insertions(+), 56 deletions(-)
>
> --
> 2.44.0
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes
2024-05-13 9:49 [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Christoph Heiss
` (4 preceding siblings ...)
2024-05-13 11:15 ` [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes Aaron Lauterer
@ 2024-06-17 7:11 ` Christoph Heiss
5 siblings, 0 replies; 8+ messages in thread
From: Christoph Heiss @ 2024-06-17 7:11 UTC (permalink / raw)
To: Proxmox VE development discussion
Ping. Still applies cleanly.
On Mon, May 13, 2024 at 11:49:08AM +0200, Christoph Heiss wrote:
> The proxmox-auto-install-assistant uses
> - glob patterns for disk matching, which can be pre-compiled for
> efficiency
> - regexes for udev property matching, which can be simplified by some
> simple prefix matching & splitting on =
>
> The latter also significantly reduces binary size due to the removing
> the regex dependency, for details see patch #4.
>
> Overall no functional changes in this series.
>
> v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063802.html
>
> Changes v1 -> v2:
> * rebased on latest master
>
> Christoph Heiss (4):
> tree-wide: run rustfmt, fix clippy warnings
> assistant: drop unused `log` dependency
> assistant: pre-compile ignored block device patterns
> assistant: avoid regex for simple prefix matching
>
> proxmox-auto-install-assistant/Cargo.toml | 2 -
> proxmox-auto-install-assistant/src/main.rs | 75 ++++++++-----------
> proxmox-auto-installer/tests/parse-answer.rs | 14 ++--
> .../src/fetch_plugins/partition.rs | 10 +--
> 4 files changed, 45 insertions(+), 56 deletions(-)
>
> --
> 2.44.0
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 8+ messages in thread