[pve-devel] [PATCH installer 0/4] assistant: clean up glob patterns & regexes

[pve-devel] [PATCH installer 0/4] assistant: clean up glob patterns & regexes

[Christoph Heiss]

The proxmox-auto-install-assistant uses
  - glob patterns for disk matching, which can be pre-compiled for
    efficiency
  - regexes for udev property matching, which can be simplified by some
    simple prefix matching & splitting on =

The latter also significantly reduces binary size due to the removing
the regex dependency, for details see patch #4.

Overall no functional changes in this series.

Christoph Heiss (4):
  tree-wide: run rustfmt, fix clippy warnings
  assistant: drop unused `log` dependency
  assistant: pre-compile ignored block device patterns
  assistant: avoid regex for simple prefix matching

 proxmox-auto-install-assistant/Cargo.toml     |  2 -
 proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
 proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
 .../src/fetch_plugins/partition.rs            | 10 +--
 4 files changed, 45 insertions(+), 56 deletions(-)

--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer 1/4] tree-wide: run rustfmt, fix clippy warnings

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 proxmox-auto-installer/tests/parse-answer.rs       | 14 +++++++-------
 .../src/fetch_plugins/partition.rs                 | 10 +++++-----
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/proxmox-auto-installer/tests/parse-answer.rs b/proxmox-auto-installer/tests/parse-answer.rs
index 4014b6d..e77a769 100644
--- a/proxmox-auto-installer/tests/parse-answer.rs
+++ b/proxmox-auto-installer/tests/parse-answer.rs
@@ -1,4 +1,4 @@
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};

 use serde_json::Value;
 use std::fs;
@@ -24,9 +24,9 @@ fn get_answer(path: PathBuf) -> Result<Answer, String> {
     Ok(answer)
 }

-fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
+fn setup_test_basic(path: &Path) -> (SetupInfo, LocaleInfo, RuntimeInfo, UdevInfo) {
     let installer_info: SetupInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("iso-info.json");

         read_json(&path)
@@ -35,7 +35,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };

     let locale_info = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("locales.json");

         read_json(&path)
@@ -44,7 +44,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };

     let mut runtime_info: RuntimeInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("run-env-info.json");

         read_json(&path)
@@ -53,7 +53,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
     };

     let udev_info: UdevInfo = {
-        let mut path = path.clone();
+        let mut path = path.to_path_buf();
         path.push("run-env-udev.json");

         read_json(&path)
@@ -71,7 +71,7 @@ fn setup_test_basic(path: &PathBuf) -> (SetupInfo, LocaleInfo, RuntimeInfo, Udev
 fn test_parse_answers() {
     let path = get_test_resource_path().unwrap();
     let (setup_info, locales, runtime_info, udev_info) = setup_test_basic(&path);
-    let mut tests_path = path.clone();
+    let mut tests_path = path;
     tests_path.push("parse_answer");
     let test_dir = fs::read_dir(tests_path.clone()).unwrap();

diff --git a/proxmox-fetch-answer/src/fetch_plugins/partition.rs b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
index 0479c8f..7213493 100644
--- a/proxmox-fetch-answer/src/fetch_plugins/partition.rs
+++ b/proxmox-fetch-answer/src/fetch_plugins/partition.rs
@@ -31,7 +31,7 @@ impl FetchFromPartition {
 }

 fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
-    let path = Path::new(search_path).join(&file_name);
+    let path = Path::new(search_path).join(file_name);
     info!("Testing partition search path {path:?}");
     match path.try_exists() {
         Ok(true) => Some(path),
@@ -51,14 +51,14 @@ fn path_exists_logged(file_name: &str, search_path: &str) -> Option<PathBuf> {
 fn scan_partlabels(partlabel: &str, search_path: &str) -> Result<PathBuf> {
     let partlabel_upper_case = partlabel.to_uppercase();
     if let Some(path) = path_exists_logged(&partlabel_upper_case, search_path) {
-            info!("Found partition with label '{partlabel_upper_case}'");
-            return Ok(path);
+        info!("Found partition with label '{partlabel_upper_case}'");
+        return Ok(path);
     }

     let partlabel_lower_case = partlabel.to_lowercase();
     if let Some(path) = path_exists_logged(&partlabel_lower_case, search_path) {
-            info!("Found partition with label '{partlabel_lower_case}'");
-            return Ok(path);
+        info!("Found partition with label '{partlabel_lower_case}'");
+        return Ok(path);
     }

     bail!("Could not detect upper or lower case labels for '{partlabel}'");
--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer 2/4] assistant: drop unused `log` dependency

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 proxmox-auto-install-assistant/Cargo.toml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index eaca7f8..0286c80 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -14,7 +14,6 @@ homepage = "https://www.proxmox.com"
 anyhow = "1.0"
 clap = { version = "4.0", features = ["derive"] }
 glob = "0.3"
-log = "0.4.20"
 proxmox-auto-installer = { path = "../proxmox-auto-installer" }
 regex = "1.7"
 serde = { version = "1.0", features = ["derive"] }
--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer 3/4] assistant: pre-compile ignored block device patterns

[Christoph Heiss]

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 proxmox-auto-install-assistant/src/main.rs | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 0debd29..906f144 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -392,13 +392,13 @@ fn inject_file_to_iso(iso: &PathBuf, file: &PathBuf, location: &str) -> Result<(
 }

 fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
-    let unwantend_block_devs = vec![
-        "ram[0-9]*",
-        "loop[0-9]*",
-        "md[0-9]*",
-        "dm-*",
-        "fd[0-9]*",
-        "sr[0-9]*",
+    let unwanted_block_devs = [
+        Pattern::new("ram[0-9]*")?,
+        Pattern::new("loop[0-9]*")?,
+        Pattern::new("md[0-9]*")?,
+        Pattern::new("dm-*")?,
+        Pattern::new("fd[0-9]*")?,
+        Pattern::new("sr[0-9]*")?,
     ];

     // compile Regex here once and not inside the loop
@@ -415,8 +415,8 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         let entry = entry.unwrap();
         let filename = entry.file_name().into_string().unwrap();

-        for p in &unwantend_block_devs {
-            if Pattern::new(p)?.matches(&filename) {
+        for p in &unwanted_block_devs {
+            if p.matches(&filename) {
                 continue 'outer;
             }
         }
--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH installer 4/4] assistant: avoid regex for simple prefix matching

[Christoph Heiss]

udev properties are very easy to parse and can be done by doing a
line-based scan and matching the prefix, splitting once for properties.
Avoids the use of regexes and signicantly reduces binary size by about
-38%(!).

Tested by comparing the output of `proxmox-auto-install-assistant
device-info`, running it before and after the changes.

Stripped binary size for release builds:
  before: 3103104 bytes ~ 2.96MiB
  after:  1935744 bytes ~ 1.85MiB

   text    data     bss     dec     hex filename
2906765  187144     537 3094446  2f37ae proxmox-auto-install-assistant-before
1871090   55736     497 1927323  1d689b proxmox-auto-install-assistant-after

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 proxmox-auto-install-assistant/Cargo.toml  |  1 -
 proxmox-auto-install-assistant/src/main.rs | 57 +++++++++-------------
 2 files changed, 24 insertions(+), 34 deletions(-)

diff --git a/proxmox-auto-install-assistant/Cargo.toml b/proxmox-auto-install-assistant/Cargo.toml
index 0286c80..766b445 100644
--- a/proxmox-auto-install-assistant/Cargo.toml
+++ b/proxmox-auto-install-assistant/Cargo.toml
@@ -15,7 +15,6 @@ anyhow = "1.0"
 clap = { version = "4.0", features = ["derive"] }
 glob = "0.3"
 proxmox-auto-installer = { path = "../proxmox-auto-installer" }
-regex = "1.7"
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 toml = "0.7"
diff --git a/proxmox-auto-install-assistant/src/main.rs b/proxmox-auto-install-assistant/src/main.rs
index 906f144..828ba76 100644
--- a/proxmox-auto-install-assistant/src/main.rs
+++ b/proxmox-auto-install-assistant/src/main.rs
@@ -1,7 +1,6 @@
 use anyhow::{bail, Result};
 use clap::{Args, Parser, Subcommand, ValueEnum};
 use glob::Pattern;
-use regex::Regex;
 use serde::Serialize;
 use std::{
     collections::BTreeMap,
@@ -401,13 +400,9 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         Pattern::new("sr[0-9]*")?,
     ];

-    // compile Regex here once and not inside the loop
-    let re_disk = Regex::new(r"(?m)^E: DEVTYPE=disk")?;
-    let re_cdrom = Regex::new(r"(?m)^E: ID_CDROM")?;
-    let re_iso9660 = Regex::new(r"(?m)^E: ID_FS_TYPE=iso9660")?;
-
-    let re_name = Regex::new(r"(?m)^N: (.*)$")?;
-    let re_props = Regex::new(r"(?m)^E: ([^=]+)=(.*)$")?;
+    const PROP_DEVTYP_PREFIX: &str = "E: DEVTYPE=";
+    const PROP_CDROM: &str = "E: ID_CDROM";
+    const PROP_ISO9660_FS: &str = "E: ID_FS_TYPE=iso9660";

     let mut disks: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();

@@ -429,30 +424,27 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
             }
         };

-        if !re_disk.is_match(&output) {
-            continue 'outer;
-        };
-        if re_cdrom.is_match(&output) {
-            continue 'outer;
-        };
-        if re_iso9660.is_match(&output) {
-            continue 'outer;
-        };
-
         let mut name = filename;
-        if let Some(cap) = re_name.captures(&output) {
-            if let Some(res) = cap.get(1) {
-                name = String::from(res.as_str());
+        let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+        for line in output.lines() {
+            if let Some(prop) = line.strip_prefix(PROP_DEVTYP_PREFIX) {
+                if prop != "disk" {
+                    continue 'outer;
+                }
             }
-        }

-        let mut udev_props: BTreeMap<String, String> = BTreeMap::new();
+            if line.starts_with(PROP_CDROM) || line.starts_with(PROP_ISO9660_FS) {
+                continue 'outer;
+            }

-        for line in output.lines() {
-            if let Some(caps) = re_props.captures(line) {
-                let key = String::from(caps.get(1).unwrap().as_str());
-                let value = String::from(caps.get(2).unwrap().as_str());
-                udev_props.insert(key, value);
+            if let Some(prop) = line.strip_prefix("N: ") {
+                name = prop.to_owned();
+            };
+
+            if let Some(prop) = line.strip_prefix("E: ") {
+                if let Some((key, val)) = prop.split_once('=') {
+                    udev_props.insert(key.to_owned(), val.to_owned());
+                }
             }
         }

@@ -462,7 +454,6 @@ fn get_disks() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
 }

 fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
-    let re_props = Regex::new(r"(?m)^E: (.*)=(.*)$")?;
     let mut nics: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();

     let links = get_nic_list()?;
@@ -480,10 +471,10 @@ fn get_nics() -> Result<BTreeMap<String, BTreeMap<String, String>>> {
         let mut udev_props: BTreeMap<String, String> = BTreeMap::new();

         for line in output.lines() {
-            if let Some(caps) = re_props.captures(line) {
-                let key = String::from(caps.get(1).unwrap().as_str());
-                let value = String::from(caps.get(2).unwrap().as_str());
-                udev_props.insert(key, value);
+            if let Some(prop) = line.strip_prefix("E: ") {
+                if let Some((key, val)) = prop.split_once('=') {
+                    udev_props.insert(key.to_owned(), val.to_owned());
+                }
             }
         }

--
2.44.0



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH installer 0/4] assistant: clean up glob patterns & regexes

[Aaron Lauterer]

it seems these patches don't apply anymore. could you please do a rebase 
on current master?

On  2024-05-07  15:21, Christoph Heiss wrote:
> The proxmox-auto-install-assistant uses
>    - glob patterns for disk matching, which can be pre-compiled for
>      efficiency
>    - regexes for udev property matching, which can be simplified by some
>      simple prefix matching & splitting on =
> 
> The latter also significantly reduces binary size due to the removing
> the regex dependency, for details see patch #4.
> 
> Overall no functional changes in this series.
> 
> Christoph Heiss (4):
>    tree-wide: run rustfmt, fix clippy warnings
>    assistant: drop unused `log` dependency
>    assistant: pre-compile ignored block device patterns
>    assistant: avoid regex for simple prefix matching
> 
>   proxmox-auto-install-assistant/Cargo.toml     |  2 -
>   proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
>   proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
>   .../src/fetch_plugins/partition.rs            | 10 +--
>   4 files changed, 45 insertions(+), 56 deletions(-)
> 
> --
> 2.44.0
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH installer 0/4] assistant: clean up glob patterns & regexes

[Christoph Heiss]

On Mon, May 13, 2024 at 11:18:51AM +0200, Aaron Lauterer wrote:
> it seems these patches don't apply anymore. could you please do a rebase on
> current master?

Sure, thanks for the notice!
Seems 1a01a01 ("assistant: keep prepared iso bootable on uefi with flash drives")
"broke" it, due to an import statement change :')

v2: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063833.html

>
> On  2024-05-07  15:21, Christoph Heiss wrote:
> > The proxmox-auto-install-assistant uses
> >    - glob patterns for disk matching, which can be pre-compiled for
> >      efficiency
> >    - regexes for udev property matching, which can be simplified by some
> >      simple prefix matching & splitting on =
> >
> > The latter also significantly reduces binary size due to the removing
> > the regex dependency, for details see patch #4.
> >
> > Overall no functional changes in this series.
> >
> > Christoph Heiss (4):
> >    tree-wide: run rustfmt, fix clippy warnings
> >    assistant: drop unused `log` dependency
> >    assistant: pre-compile ignored block device patterns
> >    assistant: avoid regex for simple prefix matching
> >
> >   proxmox-auto-install-assistant/Cargo.toml     |  2 -
> >   proxmox-auto-install-assistant/src/main.rs    | 75 ++++++++-----------
> >   proxmox-auto-installer/tests/parse-answer.rs  | 14 ++--
> >   .../src/fetch_plugins/partition.rs            | 10 +--
> >   4 files changed, 45 insertions(+), 56 deletions(-)
> >
> > --
> > 2.44.0
> >
> >
> >
> > _______________________________________________
> > pve-devel mailing list
> > pve-devel@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> >
> >


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel