[pbs-devel] [PATCH-SERIES proxmox-backup] add apt repositories support

[pbs-devel] [PATCH-SERIES proxmox-backup] add apt repositories support

[Fabian Ebner]

Basically copy-pasting the code in pve-rs and replacing instances of
"pve" with "pbs", but there are a few minor differences, need to use
String instead of &str, digest checking uses the local helper, can
use APTRepositoryHandle type directly in the API.

Can also be seen as the continuation of [0], but that series already
got it's next version, and quite a lot changed with what the UI
expects from the API calls and also a bit with the library interface,
so I just sent it as a new series.

[0]: https://lists.proxmox.com/pipermail/pbs-devel/2021-April/002596.html

Fabian Ebner (4):
  depend on proxmox-apt
  api: apt: add repositories call
  api: apt: add endpoints for adding/changing repositories
  ui: add APT repositories

 Cargo.toml                  |   1 +
 debian/control              |   1 +
 src/api2/node/apt.rs        | 218 +++++++++++++++++++++++++++++++++++-
 www/ServerAdministration.js |   8 ++
 4 files changed, 227 insertions(+), 1 deletion(-)

-- 
2.30.2

[pbs-devel] [PATCH proxmox-backup 1/4] depend on proxmox-apt

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 Cargo.toml     | 1 +
 debian/control | 1 +
 2 files changed, 2 insertions(+)

diff --git a/Cargo.toml b/Cargo.toml
index 490c17a6..78bb8fe9 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -90,6 +90,7 @@ proxmox = { version = "0.11.5", features = [ "sortable-macro", "api-macro" ] }
 #proxmox = { git = "git://git.proxmox.com/git/proxmox", version = "0.1.2", features = [ "sortable-macro", "api-macro" ] }
 #proxmox = { path = "../proxmox/proxmox", features = [ "sortable-macro", "api-macro" ] }
 proxmox-acme-rs = "0.2.1"
+proxmox-apt = "0.4.0"
 proxmox-fuse = "0.1.1"
 proxmox-http = { version = "0.2.1", features = [ "client", "http-helpers", "websocket" ] }
 #proxmox-http = { version = "0.2.0", path = "../proxmox/proxmox-http", features = [ "client", "http-helpers", "websocket" ] }
diff --git a/debian/control b/debian/control
index b710c544..a1b09340 100644
--- a/debian/control
+++ b/debian/control
@@ -43,6 +43,7 @@ Build-Depends: debhelper (>= 12),
  librust-proxmox-0.11+default-dev (>= 0.11.5-~~),
  librust-proxmox-0.11+sortable-macro-dev (>= 0.11.5-~~),
  librust-proxmox-acme-rs-0.2+default-dev (>= 0.2.1-~~),
+ librust-proxmox-apt-0.4+default-dev,
  librust-proxmox-fuse-0.1+default-dev (>= 0.1.1-~~),
  librust-proxmox-http-0.2+client-dev (>= 0.2.1-~~),
  librust-proxmox-http-0.2+default-dev (>= 0.2.1-~~),
-- 
2.30.2

[pbs-devel] [PATCH proxmox-backup 2/4] api: apt: add repositories call

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 src/api2/node/apt.rs | 70 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 69 insertions(+), 1 deletion(-)

diff --git a/src/api2/node/apt.rs b/src/api2/node/apt.rs
index 054c8813..4551b151 100644
--- a/src/api2/node/apt.rs
+++ b/src/api2/node/apt.rs
@@ -7,6 +7,9 @@ use proxmox::api::{api, RpcEnvironment, RpcEnvironmentType, Permission};
 use proxmox::api::router::{Router, SubdirMap};
 use proxmox::tools::fs::{replace_file, CreateOptions};
 
+use proxmox_apt::repositories::{
+    APTRepositoryFile, APTRepositoryFileError, APTRepositoryInfo, APTStandardRepository,
+};
 use proxmox_http::ProxyConfig;
 
 use crate::config::node;
@@ -17,7 +20,7 @@ use crate::tools::{
     subscription,
 };
 use crate::config::acl::{PRIV_SYS_AUDIT, PRIV_SYS_MODIFY};
-use crate::api2::types::{Authid, APTUpdateInfo, NODE_SCHEMA, UPID_SCHEMA};
+use crate::api2::types::{Authid, APTUpdateInfo, NODE_SCHEMA, PROXMOX_CONFIG_DIGEST_SCHEMA, UPID_SCHEMA};
 
 #[api(
     input: {
@@ -389,8 +392,73 @@ pub fn get_versions() -> Result<Vec<APTUpdateInfo>, Error> {
     Ok(packages)
 }
 
+#[api(
+    input: {
+        properties: {
+            node: {
+                schema: NODE_SCHEMA,
+            },
+        },
+    },
+    returns: {
+        type: Object,
+        description: "Result from parsing the APT repository files in /etc/apt/.",
+        properties: {
+            files: {
+                description: "List of parsed repository files.",
+                type: Array,
+                items: {
+                    type: APTRepositoryFile,
+                },
+            },
+            errors: {
+                description: "List of problematic files.",
+                type: Array,
+                items: {
+                    type: APTRepositoryFileError,
+                },
+            },
+            digest: {
+                schema: PROXMOX_CONFIG_DIGEST_SCHEMA,
+            },
+            infos: {
+                description: "List of additional information/warnings about the repositories.",
+                items: {
+                    type: APTRepositoryInfo,
+                },
+            },
+            "standard-repos": {
+                description: "List of standard repositories and their configuration status.",
+                items: {
+                    type: APTStandardRepository,
+                },
+            },
+        },
+    },
+    access: {
+        permission: &Permission::Privilege(&[], PRIV_SYS_AUDIT, false),
+    },
+)]
+/// Get APT repository information.
+pub fn get_repositories() -> Result<Value, Error> {
+    let (files, errors, digest) = proxmox_apt::repositories::repositories()?;
+    let digest = proxmox::tools::digest_to_hex(&digest);
+
+    let infos = proxmox_apt::repositories::check_repositories(&files)?;
+    let standard_repos = proxmox_apt::repositories::standard_repositories("pbs", &files);
+
+    Ok(json!({
+        "files": files,
+        "errors": errors,
+        "digest": digest,
+        "infos": infos,
+        "standard-repos": standard_repos,
+    }))
+}
+
 const SUBDIRS: SubdirMap = &[
     ("changelog", &Router::new().get(&API_METHOD_APT_GET_CHANGELOG)),
+    ("repositories", &Router::new().get(&API_METHOD_GET_REPOSITORIES)),
     ("update", &Router::new()
         .get(&API_METHOD_APT_UPDATE_AVAILABLE)
         .post(&API_METHOD_APT_UPDATE_DATABASE)
-- 
2.30.2

[pbs-devel] [PATCH proxmox-backup 3/4] api: apt: add endpoints for adding/changing repositories

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---

Could also be squashed with the previous one.

Used SYS_MODIFY as a privilege. Should this rather be superuser only?

 src/api2/node/apt.rs | 152 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 150 insertions(+), 2 deletions(-)

diff --git a/src/api2/node/apt.rs b/src/api2/node/apt.rs
index 4551b151..271277d1 100644
--- a/src/api2/node/apt.rs
+++ b/src/api2/node/apt.rs
@@ -8,7 +8,8 @@ use proxmox::api::router::{Router, SubdirMap};
 use proxmox::tools::fs::{replace_file, CreateOptions};
 
 use proxmox_apt::repositories::{
-    APTRepositoryFile, APTRepositoryFileError, APTRepositoryInfo, APTStandardRepository,
+    APTRepositoryFile, APTRepositoryFileError, APTRepositoryHandle, APTRepositoryInfo,
+    APTStandardRepository,
 };
 use proxmox_http::ProxyConfig;
 
@@ -456,9 +457,156 @@ pub fn get_repositories() -> Result<Value, Error> {
     }))
 }
 
+#[api(
+    input: {
+        properties: {
+            node: {
+                schema: NODE_SCHEMA,
+            },
+            handle: {
+                type: APTRepositoryHandle,
+            },
+            digest: {
+                schema: PROXMOX_CONFIG_DIGEST_SCHEMA,
+                optional: true,
+            },
+        },
+    },
+    protected: true,
+    access: {
+        permission: &Permission::Privilege(&[], PRIV_SYS_MODIFY, false),
+    },
+)]
+/// Add the repository identified by the `handle`.
+/// If the repository is already configured, it will be set to enabled.
+///
+/// The `digest` parameter asserts that the configuration has not been modified.
+pub fn add_repository(handle: APTRepositoryHandle, digest: Option<String>) -> Result<(), Error> {
+    let (mut files, errors, current_digest) = proxmox_apt::repositories::repositories()?;
+
+    if let Some(expected_digest) = digest {
+        let current_digest = proxmox::tools::digest_to_hex(&current_digest);
+        crate::tools::assert_if_modified(&expected_digest, &current_digest)?;
+    }
+
+    // check if it's already configured first
+    for file in files.iter_mut() {
+        for repo in file.repositories.iter_mut() {
+            if repo.is_referenced_repository(handle, "pbs") {
+                if repo.enabled {
+                    return Ok(());
+                }
+
+                repo.set_enabled(true);
+                file.write()?;
+
+                return Ok(());
+            }
+        }
+    }
+
+    let (repo, path) = proxmox_apt::repositories::get_standard_repository(handle, "pbs")?;
+
+    if let Some(error) = errors.iter().find(|error| error.path == path) {
+        bail!(
+            "unable to parse existing file {} - {}",
+            error.path,
+            error.error,
+        );
+    }
+
+    if let Some(file) = files.iter_mut().find(|file| file.path == path) {
+        file.repositories.push(repo);
+
+        file.write()?;
+    } else {
+        let mut file = match APTRepositoryFile::new(&path)? {
+            Some(file) => file,
+            None => bail!("invalid path - {}", path),
+        };
+
+        file.repositories.push(repo);
+
+        file.write()?;
+    }
+
+    Ok(())
+}
+
+#[api(
+    input: {
+        properties: {
+            node: {
+                schema: NODE_SCHEMA,
+            },
+            path: {
+                description: "Path to the containing file.",
+                type: String,
+            },
+            index: {
+                description: "Index within the file (starting from 0).",
+                type: usize,
+            },
+            enabled: {
+                description: "Whether the repository should be enabled or not.",
+                type: bool,
+                optional: true,
+            },
+            digest: {
+                schema: PROXMOX_CONFIG_DIGEST_SCHEMA,
+                optional: true,
+            },
+        },
+    },
+    protected: true,
+    access: {
+        permission: &Permission::Privilege(&[], PRIV_SYS_MODIFY, false),
+    },
+)]
+/// Change the properties of the specified repository.
+///
+/// The `digest` parameter asserts that the configuration has not been modified.
+pub fn change_repository(
+    path: String,
+    index: usize,
+    enabled: Option<bool>,
+    digest: Option<String>,
+) -> Result<(), Error> {
+    let (mut files, errors, current_digest) = proxmox_apt::repositories::repositories()?;
+
+    if let Some(expected_digest) = digest {
+        let current_digest = proxmox::tools::digest_to_hex(&current_digest);
+        crate::tools::assert_if_modified(&expected_digest, &current_digest)?;
+    }
+
+    if let Some(error) = errors.iter().find(|error| error.path == path) {
+        bail!("unable to parse file {} - {}", error.path, error.error);
+    }
+
+    if let Some(file) = files.iter_mut().find(|file| file.path == path) {
+        if let Some(repo) = file.repositories.get_mut(index) {
+            if let Some(enabled) = enabled {
+                repo.set_enabled(enabled);
+            }
+
+            file.write()?;
+        } else {
+            bail!("invalid index - {}", index);
+        }
+    } else {
+        bail!("invalid path - {}", path);
+    }
+
+    Ok(())
+}
+
 const SUBDIRS: SubdirMap = &[
     ("changelog", &Router::new().get(&API_METHOD_APT_GET_CHANGELOG)),
-    ("repositories", &Router::new().get(&API_METHOD_GET_REPOSITORIES)),
+    ("repositories", &Router::new()
+        .get(&API_METHOD_GET_REPOSITORIES)
+        .post(&API_METHOD_CHANGE_REPOSITORY)
+        .put(&API_METHOD_ADD_REPOSITORY)
+    ),
     ("update", &Router::new()
         .get(&API_METHOD_APT_UPDATE_AVAILABLE)
         .post(&API_METHOD_APT_UPDATE_DATABASE)
-- 
2.30.2

[pbs-devel] [PATCH proxmox-backup 4/4] ui: add APT repositories

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 www/ServerAdministration.js | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/www/ServerAdministration.js b/www/ServerAdministration.js
index 61b8246d..fd93c173 100644
--- a/www/ServerAdministration.js
+++ b/www/ServerAdministration.js
@@ -53,6 +53,14 @@ Ext.define('PBS.ServerAdministration', {
 	    itemId: 'updates',
 	    nodename: 'localhost',
 	},
+	{
+	    xtype: 'proxmoxNodeAPTRepositories',
+	    title: gettext('Repositories'),
+	    iconCls: 'fa fa-files-o',
+	    itemId: 'aptrepositories',
+	    nodename: 'localhost',
+	    product: 'Proxmox Backup Server',
+	},
 	{
 	    xtype: 'proxmoxJournalView',
 	    itemId: 'logs',
-- 
2.30.2

[pbs-devel] applied-series: [PATCH-SERIES proxmox-backup] add apt repositories support

[Thomas Lamprecht]

On 08.07.21 16:14, Fabian Ebner wrote:
> Basically copy-pasting the code in pve-rs and replacing instances of
> "pve" with "pbs", but there are a few minor differences, need to use
> String instead of &str, digest checking uses the local helper, can
> use APTRepositoryHandle type directly in the API.
> 
> Can also be seen as the continuation of [0], but that series already
> got it's next version, and quite a lot changed with what the UI
> expects from the API calls and also a bit with the library interface,
> so I just sent it as a new series.
> 
> [0]: https://lists.proxmox.com/pipermail/pbs-devel/2021-April/002596.html
> 
> Fabian Ebner (4):
>   depend on proxmox-apt
>   api: apt: add repositories call
>   api: apt: add endpoints for adding/changing repositories
>   ui: add APT repositories
> 
>  Cargo.toml                  |   1 +
>  debian/control              |   1 +
>  src/api2/node/apt.rs        | 218 +++++++++++++++++++++++++++++++++++-
>  www/ServerAdministration.js |   8 ++
>  4 files changed, 227 insertions(+), 1 deletion(-)
> 

Really nice to see so much code-reuse and so little code needed to get this in
here, great!

applied series, thanks!