[pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes

[pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes

[Christoph Heiss]

The proxmox-auto-install-assistant uses
  - glob patterns for disk matching, which can be pre-compiled for
    efficiency
  - regexes for udev property matching, which can be simplified by some
    simple prefix matching & splitting on =

The latter also significantly reduces binary size due to the removing
the regex dependency, for details see patch #4.

Overall no functional changes in this series.

v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063802.html

Changes v1 -> v2:
  * rebased on latest master

Christoph Heiss (4):
  tree-wide: run rustfmt, fix clippy warnings
  assistant: drop unused `log` dependency
  assistant: pre-compile ignored block device patterns
  assistant: avoid regex for simple prefix matching

 proxmox-auto-install-assistant/Cargo.toml     |  2 -
 proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
 proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
 .../src/fetch_plugins/partition.rs            | 10 +--
 4 files changed, 45 insertions(+), 56 deletions(-)

--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer v2 1/4] tree-wide: run rustfmt, fix clippy warnings

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
  * no changes

 proxmox-auto-installer/tests/parse-answer.rs       | 14 +++++++-------
 .../src/fetch_plugins/partition.rs                 | 10 +++++-----
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs
index 4014b6d..e77a769 100644
--- a/proxmox-auto-installer/tests/parse-answer.rs
+++ b/proxmox-auto-installer/tests/parse-answer.rs
@@ -1,4 +1,4 @@
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 
 use serde_json::Value;
 use std::fs;
@@ -24,9 +24,9 @@ fn get_answer(path: PathBuf) -> Result<Answer, String> {
     Ok(answer)
 }
 
-fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
+fn setup_test_basic(path: &Path) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
     let installer_info: SetupInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("iso-info.json");
 
         read_json(&path)
@@ -35,7 +35,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };
 
     let locale_info = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("locales.json");
 
         read_json(&path)
@@ -44,7 +44,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };
 
     let mut runtime_info: RuntimeInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("run-env-info.json");
 
         read_json(&path)
@@ -53,7 +53,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };
 
     let udev_info: UdevInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("run-env-udev.json");
 
         read_json(&path)
@@ -71,7 +71,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
 fn test_parse_answers() {
     let path = get_test_resource_path().unwrap();
     let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path);
-    let mut tests_path = path.clone();
+    let mut tests_path = path;
     tests_path.push("parse_answer");
     let test_dir = fs::read_dir(tests_path.clone()).unwrap();
 
diff --git a/proxmox-fetch-answer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
index 0479c8f..7213493 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -31,7 +31,7 @@ impl FetchFromPartition {
 }
 
 fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
-    let path = Path::new(search_path).join(&file_name);
+    let path = Path::new(search_path).join(file_name);
     info!("Testing partition search path {path:?}");
     match path.try_exists() {
         Ok(true) => Some(path),
@@ -51,14 +51,14 @@ fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
 fn scan_partlabels(partlabel: &str, search_path: &str) -> Result<PathBuf> {
     let partlabel_upper_case = partlabel.to_uppercase();
     if let Some(path) = path_exists_logged(&partlabel_upper_case, search_path) {
-            info!("Found partition with label '{partlabel_upper_case}'");
-            return Ok(path);
+        info!("Found partition with label '{partlabel_upper_case}'");
+        return Ok(path);
     }
 
     let partlabel_lower_case = partlabel.to_lowercase();
     if let Some(path) = path_exists_logged(&partlabel_lower_case, search_path) {
-            info!("Found partition with label '{partlabel_lower_case}'");
-            return Ok(path);
+        info!("Found partition with label '{partlabel_lower_case}'");
+        return Ok(path);
     }
 
     bail!("Could not detect upper or lower case labels for '{partlabel}'");
-- 
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer v2 2/4] assistant: drop unused `log` dependency

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
  * no changes

 proxmox-auto-install-assistant/Cargo.toml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index eaca7f8..0286c80 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -14,7 +14,6 @@ homepage = "https://www.proxmox.com"
 anyhow = "1.0"
 clap = { version = "4.0", features = ["derive"] }
 glob = "0.3"
-log = "0.4.20"
 proxmox-auto-installer = { path = "../proxmox-auto-installer" }
 regex = "1.7"
 serde = { version = "1.0", features = ["derive"] }
-- 
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer v2 3/4] assistant: pre-compile ignored block device patterns

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
  * no changes

 proxmox-auto-install-assistant/src/main.rs | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 1447175..790dbc7 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -430,13 +430,13 @@ fn get_iso_uuid(iso: &PathBuf) -> Result<String> {
 }
 
 fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
-    let unwantend_block_devs = vec![
-        "ram[0-9]*",
-        "loop[0-9]*",
-        "md[0-9]*",
-        "dm-*",
-        "fd[0-9]*",
-        "sr[0-9]*",
+    let unwanted_block_devs = [
+        Pattern::new("ram[0-9]*")?,
+        Pattern::new("loop[0-9]*")?,
+        Pattern::new("md[0-9]*")?,
+        Pattern::new("dm-*")?,
+        Pattern::new("fd[0-9]*")?,
+        Pattern::new("sr[0-9]*")?,
     ];
 
     // compile Regex here once and not inside the loop
@@ -453,8 +453,8 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         let entry = entry.unwrap();
         let filename = entry.file_name().into_string().unwrap();
 
-        for p in &unwantend_block_devs {
-            if Pattern::new(p)?.matches(&filename) {
+        for p in &unwanted_block_devs {
+            if p.matches(&filename) {
                 continue 'outer;
             }
         }
-- 
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer v2 4/4] assistant: avoid regex for simple prefix matching

[Christoph Heiss]

udev properties are very easy to parse and can be done by doing a
line-based scan and matching the prefix, splitting once for properties.
Avoids the use of regexes and signicantly reduces binary size by about
-38%(!).

Tested by comparing the output of `proxmox-auto-install-assistant
device-info`, running it before and after the changes.

Stripped binary size for release builds:
  before: 3103104 bytes ~ 2.96MiB
  after:  1935744 bytes ~ 1.85MiB

   text    data     bss     dec     hex filename
2906765  187144     537 3094446  2f37ae proxmox-auto-install-assistant-before
1871090   55736     497 1927323  1d689b proxmox-auto-install-assistant-after

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
  * rebased on latest master

 proxmox-auto-install-assistant/Cargo.toml  |  1 -
 proxmox-auto-install-assistant/src/main.rs | 57 +++++++++-------------
 2 files changed, 24 insertions(+), 34 deletions(-)

diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index 0286c80..766b445 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -15,7 +15,6 @@ anyhow = "1.0"
 clap = { version = "4.0", features = ["derive"] }
 glob = "0.3"
 proxmox-auto-installer = { path = "../proxmox-auto-installer" }
-regex = "1.7"
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 toml = "0.7"
diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 790dbc7..7c0b0c6 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -1,7 +1,6 @@
 use anyhow::{bail, format_err, Result};
 use clap::{Args, Parser, Subcommand, ValueEnum};
 use glob::Pattern;
-use regex::Regex;
 use serde::Serialize;
 use std::{
     collections::BTreeMap,
@@ -439,13 +438,9 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         Pattern::new("sr[0-9]*")?,
     ];
 
-    // compile Regex here once and not inside the loop
-    let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?;
-    let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?;
-    let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?;
-
-    let re_name = Regex::new(r"(?m)^N: (.*)$")?;
-    let re_props = Regex::new(r"(?m)^E: ([^=]+)=(.*)$")?;
+    const PROP_DEVTYP_PREFIX: &str = "E: DEVTYPE=";
+    const PROP_CDROM: &str = "E: ID_CDROM";
+    const PROP_ISO9660_FS: &str = "E: ID_FS_TYPE=iso9660";
 
     let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
 
@@ -467,30 +462,27 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
             }
         };
 
-        if !re_disk.is_match(&output) {
-            continue 'outer;
-        };
-        if re_cdrom.is_match(&output) {
-            continue 'outer;
-        };
-        if re_iso9660.is_match(&output) {
-            continue 'outer;
-        };
-
         let mut name = filename;
-        if let Some(cap) = re_name.captures(&output) {
-            if let Some(res) = cap.get(1) {
-                name = String::from(res.as_str());
+        let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+        for line in output.lines() {
+            if let Some(prop) = line.strip_prefix(PROP_DEVTYP_PREFIX) {
+                if prop != "disk" {
+                    continue 'outer;
+                }
             }
-        }
 
-        let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+            if line.starts_with(PROP_CDROM) || line.starts_with(PROP_ISO9660_FS) {
+                continue 'outer;
+            }
 
-        for line in output.lines() {
-            if let Some(caps) = re_props.captures(line) {
-                let key = String::from(caps.get(1).unwrap().as_str());
-                let value = String::from(caps.get(2).unwrap().as_str());
-                udev_props.insert(key, value);
+            if let Some(prop) = line.strip_prefix("N: ") {
+                name = prop.to_owned();
+            };
+
+            if let Some(prop) = line.strip_prefix("E: ") {
+                if let Some((key, val)) = prop.split_once('=') {
+                    udev_props.insert(key.to_owned(), val.to_owned());
+                }
             }
         }
 
@@ -500,7 +492,6 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
 }
 
 fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
-    let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
     let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
 
     let links = get_nic_list()?;
@@ -518,10 +509,10 @@ fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
 
         for line in output.lines() {
-            if let Some(caps) = re_props.captures(line) {
-                let key = String::from(caps.get(1).unwrap().as_str());
-                let value = String::from(caps.get(2).unwrap().as_str());
-                udev_props.insert(key, value);
+            if let Some(prop) = line.strip_prefix("E: ") {
+                if let Some((key, val)) = prop.split_once('=') {
+                    udev_props.insert(key.to_owned(), val.to_owned());
+                }
             }
         }
 
-- 
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes

[Aaron Lauterer]

did some tests on my local machine with the `device-info` and 
`device-match` subcommands to list and match against identifiers of 
disks and my NIC.

Thanks for taking the time to improve this code.

consider this series:
Reviewed-By: Aaron Lauterer <a.lauterer@proxmox.com>
Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>

On  2024-05-13  11:49, Christoph Heiss wrote:
> The proxmox-auto-install-assistant uses
>    - glob patterns for disk matching, which can be pre-compiled for
>      efficiency
>    - regexes for udev property matching, which can be simplified by some
>      simple prefix matching & splitting on =
> 
> The latter also significantly reduces binary size due to the removing
> the regex dependency, for details see patch #4.
> 
> Overall no functional changes in this series.
> 
> v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063802.html
> 
> Changes v1 -> v2:
>    * rebased on latest master
> 
> Christoph Heiss (4):
>    tree-wide: run rustfmt, fix clippy warnings
>    assistant: drop unused `log` dependency
>    assistant: pre-compile ignored block device patterns
>    assistant: avoid regex for simple prefix matching
> 
>   proxmox-auto-install-assistant/Cargo.toml     |  2 -
>   proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
>   proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
>   .../src/fetch_plugins/partition.rs            | 10 +--
>   4 files changed, 45 insertions(+), 56 deletions(-)
> 
> --
> 2.44.0
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH installer v2 0/4] assistant: clean up glob patterns & regexes

[Christoph Heiss]

Ping. Still applies cleanly.

On Mon, May 13, 2024 at 11:49:08AM +0200, Christoph Heiss wrote:
> The proxmox-auto-install-assistant uses
>   - glob patterns for disk matching, which can be pre-compiled for
>     efficiency
>   - regexes for udev property matching, which can be simplified by some
>     simple prefix matching & splitting on =
>
> The latter also significantly reduces binary size due to the removing
> the regex dependency, for details see patch #4.
>
> Overall no functional changes in this series.
>
> v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063802.html
>
> Changes v1 -> v2:
>   * rebased on latest master
>
> Christoph Heiss (4):
>   tree-wide: run rustfmt, fix clippy warnings
>   assistant: drop unused `log` dependency
>   assistant: pre-compile ignored block device patterns
>   assistant: avoid regex for simple prefix matching
>
>  proxmox-auto-install-assistant/Cargo.toml     |  2 -
>  proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
>  proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
>  .../src/fetch_plugins/partition.rs            | 10 +--
>  4 files changed, 45 insertions(+), 56 deletions(-)
>
> --
> 2.44.0
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH installer v2 3/4] assistant: pre-compile ignored block device patterns

[Thomas Lamprecht]

Am 13.05.24 um 11:49 schrieb Christoph Heiss:
> No functional changes.
> 
> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
> ---
> Changes v1 -> v2:
>   * no changes
> 
>  proxmox-auto-install-assistant/src/main.rs | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
>

applied this one with Aarons review tags, thanks!

The rest would need to be rebased, sorry for the wait here.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel