* [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate
@ 2024-07-09 6:18 Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 2/6] apt-api-types: use serde-plain to display/parse enums Wolfgang Bumiller
` (4 more replies)
0 siblings, 5 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
Cargo.toml | 2 +
proxmox-apt-api-types/Cargo.toml | 15 +
proxmox-apt-api-types/debian/changelog | 5 +
proxmox-apt-api-types/debian/control | 42 ++
proxmox-apt-api-types/debian/copyright | 18 +
proxmox-apt-api-types/debian/debcargo.toml | 7 +
proxmox-apt-api-types/src/lib.rs | 442 +++++++++++++++++++++
7 files changed, 531 insertions(+)
create mode 100644 proxmox-apt-api-types/Cargo.toml
create mode 100644 proxmox-apt-api-types/debian/changelog
create mode 100644 proxmox-apt-api-types/debian/control
create mode 100644 proxmox-apt-api-types/debian/copyright
create mode 100644 proxmox-apt-api-types/debian/debcargo.toml
create mode 100644 proxmox-apt-api-types/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index 48fa77b6..15556670 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -5,6 +5,7 @@ members = [
"proxmox-acme-api",
"proxmox-api-macro",
"proxmox-apt",
+ "proxmox-apt-api-types",
"proxmox-async",
"proxmox-auth-api",
"proxmox-borrow",
@@ -111,6 +112,7 @@ zstd = { version = "0.12", features = [ "bindgen" ] }
# workspace dependencies
proxmox-acme = { version = "0.5.2", path = "proxmox-acme", default-features = false }
proxmox-api-macro = { version = "1.0.8", path = "proxmox-api-macro" }
+proxmox-apt-api-types = { version = "1.0.0", path = "proxmox-apt-api-types" }
proxmox-auth-api = { version = "0.4.0", path = "proxmox-auth-api" }
proxmox-async = { version = "0.4.1", path = "proxmox-async" }
proxmox-compression = { version = "0.2.0", path = "proxmox-compression" }
diff --git a/proxmox-apt-api-types/Cargo.toml b/proxmox-apt-api-types/Cargo.toml
new file mode 100644
index 00000000..e2ab46ad
--- /dev/null
+++ b/proxmox-apt-api-types/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "proxmox-apt-api-types"
+version = "1.0.0"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+exclude.workspace = true
+description = "APT API type definitions."
+
+[dependencies]
+anyhow.workspace = true
+serde = { workspace = true, features = ["derive"] }
+proxmox-schema = { workspace = true, features = ["api-macro"] }
+proxmox-config-digest.workspace = true
diff --git a/proxmox-apt-api-types/debian/changelog b/proxmox-apt-api-types/debian/changelog
new file mode 100644
index 00000000..32221867
--- /dev/null
+++ b/proxmox-apt-api-types/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-apt-api-types (1.0.0-1) bookworm; urgency=medium
+
+ * initial release
+
+ -- Proxmox Support Team <support@proxmox.com> Thu, 27 Jun 2024 13:14:23 +0200
diff --git a/proxmox-apt-api-types/debian/control b/proxmox-apt-api-types/debian/control
new file mode 100644
index 00000000..708e8c4b
--- /dev/null
+++ b/proxmox-apt-api-types/debian/control
@@ -0,0 +1,42 @@
+Source: rust-proxmox-apt-api-types
+Section: rust
+Priority: optional
+Build-Depends: debhelper (>= 12),
+ dh-cargo (>= 25),
+ cargo:native <!nocheck>,
+ rustc:native <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-anyhow-1+default-dev <!nocheck>,
+ librust-proxmox-config-digest-0.1+default-dev <!nocheck>,
+ librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~) <!nocheck>,
+ librust-proxmox-schema-3+default-dev (>= 3.1.1-~~) <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.6.2
+Vcs-Git: git://git.proxmox.com/git/proxmox-apt.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox-apt.git
+X-Cargo-Crate: proxmox-apt-api-types
+Rules-Requires-Root: no
+
+Package: librust-proxmox-apt-api-types-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-anyhow-1+default-dev,
+ librust-proxmox-config-digest-0.1+default-dev,
+ librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~),
+ librust-proxmox-schema-3+default-dev (>= 3.1.1-~~),
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev
+Provides:
+ librust-proxmox-apt-api-types+default-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1+default-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1.0-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1.0.0-dev (= ${binary:Version}),
+ librust-proxmox-apt-api-types-1.0.0+default-dev (= ${binary:Version})
+Description: APT API type definitions - Rust source code
+ Source code for Debianized Rust crate "proxmox-apt-api-types"
diff --git a/proxmox-apt-api-types/debian/copyright b/proxmox-apt-api-types/debian/copyright
new file mode 100644
index 00000000..b227c290
--- /dev/null
+++ b/proxmox-apt-api-types/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2019 - 2024 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/proxmox-apt-api-types/debian/debcargo.toml b/proxmox-apt-api-types/debian/debcargo.toml
new file mode 100644
index 00000000..74e38540
--- /dev/null
+++ b/proxmox-apt-api-types/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox-apt.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox-apt.git"
diff --git a/proxmox-apt-api-types/src/lib.rs b/proxmox-apt-api-types/src/lib.rs
new file mode 100644
index 00000000..3b6ac9e4
--- /dev/null
+++ b/proxmox-apt-api-types/src/lib.rs
@@ -0,0 +1,442 @@
+use std::fmt::Display;
+
+use anyhow::{bail, Error};
+use serde::{Deserialize, Serialize};
+
+use proxmox_config_digest::ConfigDigest;
+use proxmox_schema::api;
+
+#[api]
+#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "lowercase")]
+pub enum APTRepositoryFileType {
+ /// One-line-style format
+ List,
+ /// DEB822-style format
+ Sources,
+}
+
+impl TryFrom<&str> for APTRepositoryFileType {
+ type Error = Error;
+
+ fn try_from(file_type: &str) -> Result<Self, Error> {
+ match file_type {
+ "list" => Ok(APTRepositoryFileType::List),
+ "sources" => Ok(APTRepositoryFileType::Sources),
+ _ => bail!("invalid file type '{file_type}'"),
+ }
+ }
+}
+
+impl Display for APTRepositoryFileType {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ APTRepositoryFileType::List => write!(f, "list"),
+ APTRepositoryFileType::Sources => write!(f, "sources"),
+ }
+ }
+}
+
+#[api]
+#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case")]
+pub enum APTRepositoryPackageType {
+ /// Debian package
+ Deb,
+ /// Debian source package
+ DebSrc,
+}
+
+impl TryFrom<&str> for APTRepositoryPackageType {
+ type Error = Error;
+
+ fn try_from(package_type: &str) -> Result<Self, Error> {
+ match package_type {
+ "deb" => Ok(APTRepositoryPackageType::Deb),
+ "deb-src" => Ok(APTRepositoryPackageType::DebSrc),
+ _ => bail!("invalid package type '{package_type}'"),
+ }
+ }
+}
+
+impl Display for APTRepositoryPackageType {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ APTRepositoryPackageType::Deb => write!(f, "deb"),
+ APTRepositoryPackageType::DebSrc => write!(f, "deb-src"),
+ }
+ }
+}
+
+#[api(
+ properties: {
+ Key: {
+ description: "Option key.",
+ type: String,
+ },
+ Values: {
+ description: "Option values.",
+ type: Array,
+ items: {
+ description: "Value.",
+ type: String,
+ },
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")] // for consistency
+/// Additional options for an APT repository.
+/// Used for both single- and mutli-value options.
+pub struct APTRepositoryOption {
+ /// Option key.
+ pub key: String,
+ /// Option value(s).
+ pub values: Vec<String>,
+}
+
+#[api(
+ properties: {
+ Types: {
+ description: "List of package types.",
+ type: Array,
+ items: {
+ type: APTRepositoryPackageType,
+ },
+ },
+ URIs: {
+ description: "List of repository URIs.",
+ type: Array,
+ items: {
+ description: "Repository URI.",
+ type: String,
+ },
+ },
+ Suites: {
+ description: "List of distributions.",
+ type: Array,
+ items: {
+ description: "Package distribution.",
+ type: String,
+ },
+ },
+ Components: {
+ description: "List of repository components.",
+ type: Array,
+ items: {
+ description: "Repository component.",
+ type: String,
+ },
+ },
+ Options: {
+ type: Array,
+ optional: true,
+ items: {
+ type: APTRepositoryOption,
+ },
+ },
+ Comment: {
+ description: "Associated comment.",
+ type: String,
+ optional: true,
+ },
+ FileType: {
+ type: APTRepositoryFileType,
+ },
+ Enabled: {
+ description: "Whether the repository is enabled or not.",
+ type: Boolean,
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")]
+/// Describes an APT repository.
+pub struct APTRepository {
+ /// List of package types.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub types: Vec<APTRepositoryPackageType>,
+
+ /// List of repository URIs.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[serde(rename = "URIs")]
+ pub uris: Vec<String>,
+
+ /// List of package distributions.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub suites: Vec<String>,
+
+ /// List of repository components.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub components: Vec<String>,
+
+ /// Additional options.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ pub options: Vec<APTRepositoryOption>,
+
+ /// Associated comment.
+ #[serde(default, skip_serializing_if = "String::is_empty")]
+ pub comment: String,
+
+ /// Format of the defining file.
+ pub file_type: APTRepositoryFileType,
+
+ /// Whether the repository is enabled or not.
+ pub enabled: bool,
+}
+
+#[api(
+ properties: {
+ "file-type": {
+ type: APTRepositoryFileType,
+ },
+ repositories: {
+ description: "List of APT repositories.",
+ type: Array,
+ items: {
+ type: APTRepository,
+ },
+ },
+ digest: {
+ type: ConfigDigest,
+ optional: true,
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+/// Represents an abstract APT repository file.
+pub struct APTRepositoryFile {
+ /// The path to the file. If None, `contents` must be set directly.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub path: Option<String>,
+
+ /// The type of the file.
+ pub file_type: APTRepositoryFileType,
+
+ /// List of repositories in the file.
+ pub repositories: Vec<APTRepository>,
+
+ /// The file content, if already parsed.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub content: Option<String>,
+
+ /// Digest of the original contents.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub digest: Option<ConfigDigest>,
+}
+
+#[api]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+/// Error type for problems with APT repository files.
+pub struct APTRepositoryFileError {
+ /// The path to the problematic file.
+ pub path: String,
+
+ /// The error message.
+ pub error: String,
+}
+
+impl Display for APTRepositoryFileError {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ write!(f, "proxmox-apt error for '{}' - {}", self.path, self.error)
+ }
+}
+
+impl std::error::Error for APTRepositoryFileError {
+ fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+ None
+ }
+}
+
+#[api]
+#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+/// Additional information for a repository.
+pub struct APTRepositoryInfo {
+ /// Path to the defining file.
+ #[serde(default, skip_serializing_if = "String::is_empty")]
+ pub path: String,
+
+ /// Index of the associated respository within the file (starting from 0).
+ pub index: usize,
+
+ /// The property from which the info originates (e.g. "Suites")
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub property: Option<String>,
+
+ /// Info kind (e.g. "warning")
+ pub kind: String,
+
+ /// Info message
+ pub message: String,
+}
+
+#[api(
+ properties: {
+ handle: {
+ description: "Handle referencing a standard repository.",
+ type: String,
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case")]
+/// Reference to a standard repository and configuration status.
+pub struct APTStandardRepository {
+ /// Handle referencing a standard repository.
+ pub handle: APTRepositoryHandle,
+
+ /// Configuration status of the associated repository, where `None` means
+ /// not configured, and `Some(bool)` indicates enabled or disabled.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub status: Option<bool>,
+
+ /// Display name of the repository.
+ pub name: String,
+
+ /// Description of the repository.
+ pub description: String,
+}
+
+#[api]
+#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case")]
+/// Handles for Proxmox repositories.
+pub enum APTRepositoryHandle {
+ /// The enterprise repository for production use.
+ Enterprise,
+ /// The repository that can be used without subscription.
+ NoSubscription,
+ /// The test repository.
+ Test,
+ /// Ceph Quincy enterprise repository.
+ CephQuincyEnterprise,
+ /// Ceph Quincy no-subscription repository.
+ CephQuincyNoSubscription,
+ /// Ceph Quincy test repository.
+ CephQuincyTest,
+ // TODO: Add separate enum for ceph releases and use something like
+ // `CephTest(CephReleaseCodename),` once the API macro supports it.
+ /// Ceph Reef enterprise repository.
+ CephReefEnterprise,
+ /// Ceph Reef no-subscription repository.
+ CephReefNoSubscription,
+ /// Ceph Reef test repository.
+ CephReefTest,
+}
+
+#[api()]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")]
+/// Describes a package for which an update is available.
+pub struct APTUpdateInfo {
+ /// Package name
+ pub package: String,
+ /// Package title
+ pub title: String,
+ /// Package architecture
+ pub arch: String,
+ /// Human readable package description
+ pub description: String,
+ /// New version to be updated to
+ pub version: String,
+ /// Old version currently installed
+ pub old_version: String,
+ /// Package origin
+ pub origin: String,
+ /// Package priority in human-readable form
+ pub priority: String,
+ /// Package section
+ pub section: String,
+ /// Custom extra field for additional package information
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub extra_info: Option<String>,
+}
+
+#[api(
+ properties: {
+ notify: {
+ default: false,
+ optional: true,
+ },
+ quiet: {
+ default: false,
+ optional: true,
+ },
+ }
+)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+/// Options for APT update
+pub struct APTUpdateOptions {
+ /// Send notification mail about new package updates available to the email
+ /// address configured for 'root@pam').
+ pub notify: Option<bool>,
+ /// Only produces output suitable for logging, omitting progress indicators.
+ pub quiet: Option<bool>,
+}
+
+#[api(
+ properties: {
+ files: {
+ type: Array,
+ items: {
+ type: APTRepositoryFile,
+ },
+ },
+ errors: {
+ type: Array,
+ items: {
+ type: APTRepositoryFileError,
+ },
+ },
+ infos: {
+ type: Array,
+ items: {
+ type: APTRepositoryInfo,
+ },
+ },
+ "standard-repos": {
+ type: Array,
+ items: {
+ type: APTStandardRepository,
+ },
+ },
+ digest: {
+ type: ConfigDigest,
+ },
+ },
+)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+/// Result from parsing the APT repository files in /etc/apt/.
+pub struct APTRepositoriesResult {
+ /// List of problematic files.
+ pub errors: Vec<APTRepositoryFileError>,
+ /// List of standard repositories and their configuration status.
+ pub standard_repos: Vec<APTStandardRepository>,
+ /// List of additional information/warnings about the repositories
+ pub infos: Vec<APTRepositoryInfo>,
+ /// List of parsed repository files.
+ pub files: Vec<APTRepositoryFile>,
+ pub digest: ConfigDigest,
+}
+
+#[api()]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+/// Options for the get changelog API.
+pub struct APTGetChangelogOptions {
+ /// Package name to get changelog of.
+ pub name: String,
+ /// Package version to get changelog of. Omit to use candidate version.
+ pub version: Option<String>,
+}
+
+#[api()]
+#[derive(Debug, Clone, Serialize, Deserialize)]
+/// Options for the change repository API call
+pub struct APTChangeRepositoryOptions {
+ /// Whether the repository should be enabled or not.
+ pub enabled: Option<bool>,
+}
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox 2/6] apt-api-types: use serde-plain to display/parse enums
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
@ 2024-07-09 6:18 ` Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 3/6] apt: avoid direct impl on api types (use traits instead) Wolfgang Bumiller
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
proxmox-apt-api-types/Cargo.toml | 2 +-
proxmox-apt-api-types/debian/control | 8 ++---
proxmox-apt-api-types/src/lib.rs | 48 ++++------------------------
3 files changed, 12 insertions(+), 46 deletions(-)
diff --git a/proxmox-apt-api-types/Cargo.toml b/proxmox-apt-api-types/Cargo.toml
index e2ab46ad..e6482146 100644
--- a/proxmox-apt-api-types/Cargo.toml
+++ b/proxmox-apt-api-types/Cargo.toml
@@ -9,7 +9,7 @@ exclude.workspace = true
description = "APT API type definitions."
[dependencies]
-anyhow.workspace = true
serde = { workspace = true, features = ["derive"] }
+serde_plain.workspace = true
proxmox-schema = { workspace = true, features = ["api-macro"] }
proxmox-config-digest.workspace = true
diff --git a/proxmox-apt-api-types/debian/control b/proxmox-apt-api-types/debian/control
index 708e8c4b..5b50e114 100644
--- a/proxmox-apt-api-types/debian/control
+++ b/proxmox-apt-api-types/debian/control
@@ -6,12 +6,12 @@ Build-Depends: debhelper (>= 12),
cargo:native <!nocheck>,
rustc:native <!nocheck>,
libstd-rust-dev <!nocheck>,
- librust-anyhow-1+default-dev <!nocheck>,
librust-proxmox-config-digest-0.1+default-dev <!nocheck>,
librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~) <!nocheck>,
librust-proxmox-schema-3+default-dev (>= 3.1.1-~~) <!nocheck>,
librust-serde-1+default-dev <!nocheck>,
- librust-serde-1+derive-dev <!nocheck>
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-plain-1+default-dev <!nocheck>
Maintainer: Proxmox Support Team <support@proxmox.com>
Standards-Version: 4.6.2
Vcs-Git: git://git.proxmox.com/git/proxmox-apt.git
@@ -24,12 +24,12 @@ Architecture: any
Multi-Arch: same
Depends:
${misc:Depends},
- librust-anyhow-1+default-dev,
librust-proxmox-config-digest-0.1+default-dev,
librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~),
librust-proxmox-schema-3+default-dev (>= 3.1.1-~~),
librust-serde-1+default-dev,
- librust-serde-1+derive-dev
+ librust-serde-1+derive-dev,
+ librust-serde-plain-1+default-dev
Provides:
librust-proxmox-apt-api-types+default-dev (= ${binary:Version}),
librust-proxmox-apt-api-types-1-dev (= ${binary:Version}),
diff --git a/proxmox-apt-api-types/src/lib.rs b/proxmox-apt-api-types/src/lib.rs
index 3b6ac9e4..80d5ec4b 100644
--- a/proxmox-apt-api-types/src/lib.rs
+++ b/proxmox-apt-api-types/src/lib.rs
@@ -1,6 +1,5 @@
use std::fmt::Display;
-use anyhow::{bail, Error};
use serde::{Deserialize, Serialize};
use proxmox_config_digest::ConfigDigest;
@@ -16,26 +15,8 @@ pub enum APTRepositoryFileType {
Sources,
}
-impl TryFrom<&str> for APTRepositoryFileType {
- type Error = Error;
-
- fn try_from(file_type: &str) -> Result<Self, Error> {
- match file_type {
- "list" => Ok(APTRepositoryFileType::List),
- "sources" => Ok(APTRepositoryFileType::Sources),
- _ => bail!("invalid file type '{file_type}'"),
- }
- }
-}
-
-impl Display for APTRepositoryFileType {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- match self {
- APTRepositoryFileType::List => write!(f, "list"),
- APTRepositoryFileType::Sources => write!(f, "sources"),
- }
- }
-}
+serde_plain::derive_display_from_serialize!(APTRepositoryFileType);
+serde_plain::derive_fromstr_from_deserialize!(APTRepositoryFileType);
#[api]
#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
@@ -47,26 +28,8 @@ pub enum APTRepositoryPackageType {
DebSrc,
}
-impl TryFrom<&str> for APTRepositoryPackageType {
- type Error = Error;
-
- fn try_from(package_type: &str) -> Result<Self, Error> {
- match package_type {
- "deb" => Ok(APTRepositoryPackageType::Deb),
- "deb-src" => Ok(APTRepositoryPackageType::DebSrc),
- _ => bail!("invalid package type '{package_type}'"),
- }
- }
-}
-
-impl Display for APTRepositoryPackageType {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- match self {
- APTRepositoryPackageType::Deb => write!(f, "deb"),
- APTRepositoryPackageType::DebSrc => write!(f, "deb-src"),
- }
- }
-}
+serde_plain::derive_display_from_serialize!(APTRepositoryPackageType);
+serde_plain::derive_fromstr_from_deserialize!(APTRepositoryPackageType);
#[api(
properties: {
@@ -327,6 +290,9 @@ pub enum APTRepositoryHandle {
CephReefTest,
}
+serde_plain::derive_display_from_serialize!(APTRepositoryHandle);
+serde_plain::derive_fromstr_from_deserialize!(APTRepositoryHandle);
+
#[api()]
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "PascalCase")]
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox 3/6] apt: avoid direct impl on api types (use traits instead)
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 2/6] apt-api-types: use serde-plain to display/parse enums Wolfgang Bumiller
@ 2024-07-09 6:18 ` Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 4/6] apt: use api types from apt-api-types crate Wolfgang Bumiller
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
So that we can use api types from expternal crate proxmox-apt-api-types.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
proxmox-apt/src/repositories/file.rs | 68 ++++++++++++-------
.../src/repositories/file/list_parser.rs | 4 +-
.../src/repositories/file/sources_parser.rs | 1 +
proxmox-apt/src/repositories/mod.rs | 3 +
proxmox-apt/src/repositories/repository.rs | 60 ++++++++++------
proxmox-apt/src/repositories/standard.rs | 38 +++++++----
proxmox-apt/tests/repositories.rs | 3 +
7 files changed, 118 insertions(+), 59 deletions(-)
diff --git a/proxmox-apt/src/repositories/file.rs b/proxmox-apt/src/repositories/file.rs
index 0d21ce3c..086abf49 100644
--- a/proxmox-apt/src/repositories/file.rs
+++ b/proxmox-apt/src/repositories/file.rs
@@ -9,6 +9,8 @@ use crate::repositories::repository::{
APTRepository, APTRepositoryFileType, APTRepositoryPackageType,
};
+use crate::repositories::repository::APTRepositoryImpl;
+
use proxmox_schema::api;
mod list_parser;
@@ -116,13 +118,46 @@ pub struct APTRepositoryInfo {
pub message: String,
}
-impl APTRepositoryFile {
+pub trait APTRepositoryFileImpl {
/// Creates a new `APTRepositoryFile` without parsing.
///
/// If the file is hidden, the path points to a directory, or the extension
/// is usually ignored by APT (e.g. `.orig`), `Ok(None)` is returned, while
/// invalid file names yield an error.
- pub fn new<P: AsRef<Path>>(path: P) -> Result<Option<Self>, APTRepositoryFileError> {
+ fn new<P: AsRef<Path>>(path: P) -> Result<Option<APTRepositoryFile>, APTRepositoryFileError>;
+
+ fn with_content(content: String, content_type: APTRepositoryFileType) -> Self;
+
+ /// Check if the file exists.
+ fn exists(&self) -> bool;
+
+ fn read_with_digest(&self) -> Result<(Vec<u8>, [u8; 32]), APTRepositoryFileError>;
+
+ /// Create an `APTRepositoryFileError`.
+ fn err(&self, error: Error) -> APTRepositoryFileError;
+
+ /// Parses the APT repositories configured in the file on disk, including
+ /// disabled ones.
+ ///
+ /// Resets the current repositories and digest, even on failure.
+ fn parse(&mut self) -> Result<(), APTRepositoryFileError>;
+
+ /// Writes the repositories to the file on disk.
+ ///
+ /// If a digest is provided, checks that the current content of the file still
+ /// produces the same one.
+ fn write(&self) -> Result<(), APTRepositoryFileError>;
+
+ /// Checks if old or unstable suites are configured and that the Debian security repository
+ /// has the correct suite. Also checks that the `stable` keyword is not used.
+ fn check_suites(&self, current_codename: DebianCodename) -> Vec<APTRepositoryInfo>;
+
+ /// Checks for official URIs.
+ fn check_uris(&self) -> Vec<APTRepositoryInfo>;
+}
+
+impl APTRepositoryFileImpl for APTRepositoryFile {
+ fn new<P: AsRef<Path>>(path: P) -> Result<Option<Self>, APTRepositoryFileError> {
let path: PathBuf = path.as_ref().to_path_buf();
let new_err = |path_string: String, err: &str| APTRepositoryFileError {
@@ -197,7 +232,7 @@ impl APTRepositoryFile {
}))
}
- pub fn with_content(content: String, content_type: APTRepositoryFileType) -> Self {
+ fn with_content(content: String, content_type: APTRepositoryFileType) -> Self {
Self {
file_type: content_type,
content: Some(content),
@@ -207,8 +242,7 @@ impl APTRepositoryFile {
}
}
- /// Check if the file exists.
- pub fn exists(&self) -> bool {
+ fn exists(&self) -> bool {
if let Some(path) = &self.path {
PathBuf::from(path).exists()
} else {
@@ -216,7 +250,7 @@ impl APTRepositoryFile {
}
}
- pub fn read_with_digest(&self) -> Result<(Vec<u8>, [u8; 32]), APTRepositoryFileError> {
+ fn read_with_digest(&self) -> Result<(Vec<u8>, [u8; 32]), APTRepositoryFileError> {
if let Some(path) = &self.path {
let content = std::fs::read(path).map_err(|err| self.err(format_err!("{}", err)))?;
let digest = openssl::sha::sha256(&content);
@@ -233,19 +267,14 @@ impl APTRepositoryFile {
}
}
- /// Create an `APTRepositoryFileError`.
- pub fn err(&self, error: Error) -> APTRepositoryFileError {
+ fn err(&self, error: Error) -> APTRepositoryFileError {
APTRepositoryFileError {
path: self.path.clone().unwrap_or_default(),
error: error.to_string(),
}
}
- /// Parses the APT repositories configured in the file on disk, including
- /// disabled ones.
- ///
- /// Resets the current repositories and digest, even on failure.
- pub fn parse(&mut self) -> Result<(), APTRepositoryFileError> {
+ fn parse(&mut self) -> Result<(), APTRepositoryFileError> {
self.repositories.clear();
self.digest = None;
@@ -269,11 +298,7 @@ impl APTRepositoryFile {
Ok(())
}
- /// Writes the repositories to the file on disk.
- ///
- /// If a digest is provided, checks that the current content of the file still
- /// produces the same one.
- pub fn write(&self) -> Result<(), APTRepositoryFileError> {
+ fn write(&self) -> Result<(), APTRepositoryFileError> {
let path = match &self.path {
Some(path) => path,
None => {
@@ -336,9 +361,7 @@ impl APTRepositoryFile {
Ok(())
}
- /// Checks if old or unstable suites are configured and that the Debian security repository
- /// has the correct suite. Also checks that the `stable` keyword is not used.
- pub fn check_suites(&self, current_codename: DebianCodename) -> Vec<APTRepositoryInfo> {
+ fn check_suites(&self, current_codename: DebianCodename) -> Vec<APTRepositoryInfo> {
let mut infos = vec![];
let path = match &self.path {
@@ -427,8 +450,7 @@ impl APTRepositoryFile {
infos
}
- /// Checks for official URIs.
- pub fn check_uris(&self) -> Vec<APTRepositoryInfo> {
+ fn check_uris(&self) -> Vec<APTRepositoryInfo> {
let mut infos = vec![];
let path = match &self.path {
diff --git a/proxmox-apt/src/repositories/file/list_parser.rs b/proxmox-apt/src/repositories/file/list_parser.rs
index 0edeea7f..93bbcc12 100644
--- a/proxmox-apt/src/repositories/file/list_parser.rs
+++ b/proxmox-apt/src/repositories/file/list_parser.rs
@@ -3,9 +3,9 @@ use std::iter::Iterator;
use anyhow::{bail, format_err, Error};
-use crate::repositories::{APTRepository, APTRepositoryFileType, APTRepositoryOption};
-
use super::APTRepositoryParser;
+use crate::repositories::APTRepositoryImpl;
+use crate::repositories::{APTRepository, APTRepositoryFileType, APTRepositoryOption};
// TODO convert %-escape characters. Also adapt printing back accordingly,
// because at least '%' needs to be re-escaped when printing.
diff --git a/proxmox-apt/src/repositories/file/sources_parser.rs b/proxmox-apt/src/repositories/file/sources_parser.rs
index 9424bbe2..213db9d6 100644
--- a/proxmox-apt/src/repositories/file/sources_parser.rs
+++ b/proxmox-apt/src/repositories/file/sources_parser.rs
@@ -3,6 +3,7 @@ use std::iter::Iterator;
use anyhow::{bail, Error};
+use crate::repositories::APTRepositoryImpl;
use crate::repositories::{
APTRepository, APTRepositoryFileType, APTRepositoryOption, APTRepositoryPackageType,
};
diff --git a/proxmox-apt/src/repositories/mod.rs b/proxmox-apt/src/repositories/mod.rs
index 26e4192c..014f1820 100644
--- a/proxmox-apt/src/repositories/mod.rs
+++ b/proxmox-apt/src/repositories/mod.rs
@@ -4,17 +4,20 @@ use std::path::PathBuf;
use anyhow::{bail, Error};
mod repository;
+pub use repository::APTRepositoryImpl;
pub use repository::{
APTRepository, APTRepositoryFileType, APTRepositoryOption, APTRepositoryPackageType,
};
mod file;
+pub use file::APTRepositoryFileImpl;
pub use file::{APTRepositoryFile, APTRepositoryFileError, APTRepositoryInfo};
mod release;
pub use release::{get_current_release_codename, DebianCodename};
mod standard;
+pub use standard::APTRepositoryHandleImpl;
pub use standard::{APTRepositoryHandle, APTStandardRepository};
const APT_SOURCES_LIST_FILENAME: &str = "/etc/apt/sources.list";
diff --git a/proxmox-apt/src/repositories/repository.rs b/proxmox-apt/src/repositories/repository.rs
index 91bd64f2..a07db7cb 100644
--- a/proxmox-apt/src/repositories/repository.rs
+++ b/proxmox-apt/src/repositories/repository.rs
@@ -8,6 +8,7 @@ use serde::{Deserialize, Serialize};
use proxmox_schema::api;
use crate::repositories::standard::APTRepositoryHandle;
+use crate::repositories::standard::APTRepositoryHandleImpl;
#[api]
#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
@@ -188,9 +189,41 @@ pub struct APTRepository {
pub enabled: bool,
}
-impl APTRepository {
+pub trait APTRepositoryImpl {
/// Crates an empty repository.
- pub fn new(file_type: APTRepositoryFileType) -> Self {
+ fn new(file_type: APTRepositoryFileType) -> Self;
+
+ /// Changes the `enabled` flag and makes sure the `Enabled` option for
+ /// `APTRepositoryPackageType::Sources` repositories is updated too.
+ fn set_enabled(&mut self, enabled: bool);
+
+ /// Makes sure that all basic properties of a repository are present and not obviously invalid.
+ fn basic_check(&self) -> Result<(), Error>;
+
+ /// Checks if the repository is the one referenced by the handle.
+ fn is_referenced_repository(
+ &self,
+ handle: APTRepositoryHandle,
+ product: &str,
+ suite: &str,
+ ) -> bool;
+
+ /// Guess the origin from the repository's URIs.
+ ///
+ /// Intended to be used as a fallback for get_cached_origin.
+ fn origin_from_uris(&self) -> Option<String>;
+
+ /// Get the `Origin:` value from a cached InRelease file.
+ fn get_cached_origin(&self) -> Result<Option<String>, Error>;
+
+ /// Writes a repository in the corresponding format followed by a blank.
+ ///
+ /// Expects that `basic_check()` for the repository was successful.
+ fn write(&self, w: &mut dyn Write) -> Result<(), Error>;
+}
+
+impl APTRepositoryImpl for APTRepository {
+ fn new(file_type: APTRepositoryFileType) -> Self {
Self {
types: vec![],
uris: vec![],
@@ -203,9 +236,7 @@ impl APTRepository {
}
}
- /// Changes the `enabled` flag and makes sure the `Enabled` option for
- /// `APTRepositoryPackageType::Sources` repositories is updated too.
- pub fn set_enabled(&mut self, enabled: bool) {
+ fn set_enabled(&mut self, enabled: bool) {
self.enabled = enabled;
if self.file_type == APTRepositoryFileType::Sources {
@@ -226,8 +257,7 @@ impl APTRepository {
}
}
- /// Makes sure that all basic properties of a repository are present and not obviously invalid.
- pub fn basic_check(&self) -> Result<(), Error> {
+ fn basic_check(&self) -> Result<(), Error> {
if self.types.is_empty() {
bail!("missing package type(s)");
}
@@ -267,8 +297,7 @@ impl APTRepository {
Ok(())
}
- /// Checks if the repository is the one referenced by the handle.
- pub fn is_referenced_repository(
+ fn is_referenced_repository(
&self,
handle: APTRepositoryHandle,
product: &str,
@@ -299,10 +328,7 @@ impl APTRepository {
&& found_component
}
- /// Guess the origin from the repository's URIs.
- ///
- /// Intended to be used as a fallback for get_cached_origin.
- pub fn origin_from_uris(&self) -> Option<String> {
+ fn origin_from_uris(&self) -> Option<String> {
for uri in self.uris.iter() {
if let Some(host) = host_from_uri(uri) {
if host == "proxmox.com" || host.ends_with(".proxmox.com") {
@@ -318,8 +344,7 @@ impl APTRepository {
None
}
- /// Get the `Origin:` value from a cached InRelease file.
- pub fn get_cached_origin(&self) -> Result<Option<String>, Error> {
+ fn get_cached_origin(&self) -> Result<Option<String>, Error> {
for uri in self.uris.iter() {
for suite in self.suites.iter() {
let mut file = release_filename(uri, suite, false);
@@ -353,10 +378,7 @@ impl APTRepository {
Ok(None)
}
- /// Writes a repository in the corresponding format followed by a blank.
- ///
- /// Expects that `basic_check()` for the repository was successful.
- pub fn write(&self, w: &mut dyn Write) -> Result<(), Error> {
+ fn write(&self, w: &mut dyn Write) -> Result<(), Error> {
match self.file_type {
APTRepositoryFileType::List => write_one_line(self, w),
APTRepositoryFileType::Sources => write_stanza(self, w),
diff --git a/proxmox-apt/src/repositories/standard.rs b/proxmox-apt/src/repositories/standard.rs
index 29cc7885..7858fac4 100644
--- a/proxmox-apt/src/repositories/standard.rs
+++ b/proxmox-apt/src/repositories/standard.rs
@@ -111,9 +111,26 @@ impl Display for APTRepositoryHandle {
}
}
-impl APTRepositoryHandle {
+pub trait APTRepositoryHandleImpl {
/// Get the description for the repository.
- pub fn description(self) -> String {
+ fn description(self) -> String;
+ /// Get the display name of the repository.
+ fn name(self) -> String;
+ /// Get the standard file path for the repository referenced by the handle.
+ fn path(self, product: &str) -> String;
+ /// Get package type, possible URIs and the component associated with the handle.
+ ///
+ /// The first URI is the preferred one.
+ fn info(self, product: &str) -> (APTRepositoryPackageType, Vec<String>, String);
+ /// Get the standard repository referenced by the handle.
+ ///
+ /// An URI in the result is not '/'-terminated (under the assumption that no valid
+ /// product name is).
+ fn to_repository(self, product: &str, suite: &str) -> APTRepository;
+}
+
+impl APTRepositoryHandleImpl for APTRepositoryHandle {
+ fn description(self) -> String {
match self {
APTRepositoryHandle::Enterprise => {
"This is the default, stable, and recommended repository, available for all \
@@ -155,8 +172,7 @@ impl APTRepositoryHandle {
.to_string()
}
- /// Get the display name of the repository.
- pub fn name(self) -> String {
+ fn name(self) -> String {
match self {
APTRepositoryHandle::Enterprise => "Enterprise",
APTRepositoryHandle::NoSubscription => "No-Subscription",
@@ -171,8 +187,7 @@ impl APTRepositoryHandle {
.to_string()
}
- /// Get the standard file path for the repository referenced by the handle.
- pub fn path(self, product: &str) -> String {
+ fn path(self, product: &str) -> String {
match self {
APTRepositoryHandle::Enterprise => {
format!("/etc/apt/sources.list.d/{}-enterprise.list", product)
@@ -188,10 +203,7 @@ impl APTRepositoryHandle {
}
}
- /// Get package type, possible URIs and the component associated with the handle.
- ///
- /// The first URI is the preferred one.
- pub fn info(self, product: &str) -> (APTRepositoryPackageType, Vec<String>, String) {
+ fn info(self, product: &str) -> (APTRepositoryPackageType, Vec<String>, String) {
match self {
APTRepositoryHandle::Enterprise => (
APTRepositoryPackageType::Deb,
@@ -259,11 +271,7 @@ impl APTRepositoryHandle {
}
}
- /// Get the standard repository referenced by the handle.
- ///
- /// An URI in the result is not '/'-terminated (under the assumption that no valid
- /// product name is).
- pub fn to_repository(self, product: &str, suite: &str) -> APTRepository {
+ fn to_repository(self, product: &str, suite: &str) -> APTRepository {
let (package_type, uris, component) = self.info(product);
APTRepository {
diff --git a/proxmox-apt/tests/repositories.rs b/proxmox-apt/tests/repositories.rs
index ae92e51c..37d665bf 100644
--- a/proxmox-apt/tests/repositories.rs
+++ b/proxmox-apt/tests/repositories.rs
@@ -8,6 +8,9 @@ use proxmox_apt::repositories::{
check_repositories, get_current_release_codename, standard_repositories, APTRepositoryFile,
APTRepositoryHandle, APTRepositoryInfo, APTStandardRepository, DebianCodename,
};
+use proxmox_apt::repositories::{
+ APTRepositoryFileImpl, APTRepositoryHandleImpl, APTRepositoryImpl,
+};
fn create_clean_directory(path: &PathBuf) -> Result<(), Error> {
match std::fs::remove_dir_all(path) {
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox 4/6] apt: use api types from apt-api-types crate
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 2/6] apt-api-types: use serde-plain to display/parse enums Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 3/6] apt: avoid direct impl on api types (use traits instead) Wolfgang Bumiller
@ 2024-07-09 6:18 ` Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 5/6] apt: add cache feature Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config Wolfgang Bumiller
4 siblings, 0 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
proxmox-apt/Cargo.toml | 5 +-
proxmox-apt/debian/control | 10 +-
proxmox-apt/src/repositories/file.rs | 119 ++---------
.../src/repositories/file/list_parser.rs | 2 +-
.../src/repositories/file/sources_parser.rs | 2 +-
proxmox-apt/src/repositories/mod.rs | 37 ++--
proxmox-apt/src/repositories/repository.rs | 187 +-----------------
proxmox-apt/src/repositories/standard.rs | 107 +---------
proxmox-apt/tests/repositories.rs | 24 +--
9 files changed, 65 insertions(+), 428 deletions(-)
diff --git a/proxmox-apt/Cargo.toml b/proxmox-apt/Cargo.toml
index 34573543..bbd4ff89 100644
--- a/proxmox-apt/Cargo.toml
+++ b/proxmox-apt/Cargo.toml
@@ -8,7 +8,7 @@ license.workspace = true
repository.workspace = true
homepage.workspace = true
-exclude = [ "debian" ]
+exclude = ["debian"]
[dependencies]
anyhow.workspace = true
@@ -20,4 +20,5 @@ serde_json.workspace = true
rfc822-like = "0.2.1"
-proxmox-schema = { workspace = true, features = [ "api-macro" ] }
+proxmox-apt-api-types.workspace = true
+proxmox-config-digest = { workspace = true, features = ["openssl"] }
diff --git a/proxmox-apt/debian/control b/proxmox-apt/debian/control
index c3248212..347631e6 100644
--- a/proxmox-apt/debian/control
+++ b/proxmox-apt/debian/control
@@ -10,8 +10,9 @@ Build-Depends: debhelper (>= 12),
librust-hex-0.4+default-dev <!nocheck>,
librust-once-cell-1+default-dev (>= 1.3.1-~~) <!nocheck>,
librust-openssl-0.10+default-dev <!nocheck>,
- librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~) <!nocheck>,
- librust-proxmox-schema-3+default-dev (>= 3.1.1-~~) <!nocheck>,
+ librust-proxmox-apt-api-types-1+default-dev <!nocheck>,
+ librust-proxmox-config-digest-0.1+default-dev <!nocheck>,
+ librust-proxmox-config-digest-0.1+openssl-dev <!nocheck>,
librust-rfc822-like-0.2+default-dev (>= 0.2.1-~~) <!nocheck>,
librust-serde-1+default-dev <!nocheck>,
librust-serde-1+derive-dev <!nocheck>,
@@ -33,8 +34,9 @@ Depends:
librust-hex-0.4+default-dev,
librust-once-cell-1+default-dev (>= 1.3.1-~~),
librust-openssl-0.10+default-dev,
- librust-proxmox-schema-3+api-macro-dev (>= 3.1.1-~~),
- librust-proxmox-schema-3+default-dev (>= 3.1.1-~~),
+ librust-proxmox-apt-api-types-1+default-dev,
+ librust-proxmox-config-digest-0.1+default-dev,
+ librust-proxmox-config-digest-0.1+openssl-dev,
librust-rfc822-like-0.2+default-dev (>= 0.2.1-~~),
librust-serde-1+default-dev,
librust-serde-1+derive-dev,
diff --git a/proxmox-apt/src/repositories/file.rs b/proxmox-apt/src/repositories/file.rs
index 086abf49..21f612ef 100644
--- a/proxmox-apt/src/repositories/file.rs
+++ b/proxmox-apt/src/repositories/file.rs
@@ -1,123 +1,29 @@
-use std::fmt::Display;
use std::path::{Path, PathBuf};
use anyhow::{format_err, Error};
-use serde::{Deserialize, Serialize};
use crate::repositories::release::DebianCodename;
-use crate::repositories::repository::{
- APTRepository, APTRepositoryFileType, APTRepositoryPackageType,
+use proxmox_apt_api_types::{
+ APTRepository, APTRepositoryFile, APTRepositoryFileError, APTRepositoryFileType,
+ APTRepositoryInfo, APTRepositoryPackageType,
};
use crate::repositories::repository::APTRepositoryImpl;
-use proxmox_schema::api;
-
mod list_parser;
use list_parser::APTListFileParser;
mod sources_parser;
use sources_parser::APTSourcesFileParser;
+use proxmox_config_digest::ConfigDigest;
+
trait APTRepositoryParser {
/// Parse all repositories including the disabled ones and push them onto
/// the provided vector.
fn parse_repositories(&mut self) -> Result<Vec<APTRepository>, Error>;
}
-#[api(
- properties: {
- "file-type": {
- type: APTRepositoryFileType,
- },
- repositories: {
- description: "List of APT repositories.",
- type: Array,
- items: {
- type: APTRepository,
- },
- },
- digest: {
- description: "Digest for the content of the file.",
- optional: true,
- type: Array,
- items: {
- description: "Digest byte.",
- type: u8,
- },
- },
- },
-)]
-#[derive(Debug, Clone, Serialize, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-/// Represents an abstract APT repository file.
-pub struct APTRepositoryFile {
- /// The path to the file. If None, `contents` must be set directly.
- #[serde(skip_serializing_if = "Option::is_none")]
- pub path: Option<String>,
-
- /// The type of the file.
- pub file_type: APTRepositoryFileType,
-
- /// List of repositories in the file.
- pub repositories: Vec<APTRepository>,
-
- /// The file content, if already parsed.
- #[serde(skip_serializing_if = "Option::is_none")]
- pub content: Option<String>,
-
- /// Digest of the original contents.
- #[serde(skip_serializing_if = "Option::is_none")]
- pub digest: Option<[u8; 32]>,
-}
-
-#[api]
-#[derive(Debug, Clone, Serialize, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-/// Error type for problems with APT repository files.
-pub struct APTRepositoryFileError {
- /// The path to the problematic file.
- pub path: String,
-
- /// The error message.
- pub error: String,
-}
-
-impl Display for APTRepositoryFileError {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- write!(f, "proxmox-apt error for '{}' - {}", self.path, self.error)
- }
-}
-
-impl std::error::Error for APTRepositoryFileError {
- fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
- None
- }
-}
-
-#[api]
-#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-/// Additional information for a repository.
-pub struct APTRepositoryInfo {
- /// Path to the defining file.
- #[serde(default, skip_serializing_if = "String::is_empty")]
- pub path: String,
-
- /// Index of the associated respository within the file (starting from 0).
- pub index: usize,
-
- /// The property from which the info originates (e.g. "Suites")
- #[serde(skip_serializing_if = "Option::is_none")]
- pub property: Option<String>,
-
- /// Info kind (e.g. "warning")
- pub kind: String,
-
- /// Info message
- pub message: String,
-}
-
pub trait APTRepositoryFileImpl {
/// Creates a new `APTRepositoryFile` without parsing.
///
@@ -131,7 +37,7 @@ pub trait APTRepositoryFileImpl {
/// Check if the file exists.
fn exists(&self) -> bool;
- fn read_with_digest(&self) -> Result<(Vec<u8>, [u8; 32]), APTRepositoryFileError>;
+ fn read_with_digest(&self) -> Result<(Vec<u8>, ConfigDigest), APTRepositoryFileError>;
/// Create an `APTRepositoryFileError`.
fn err(&self, error: Error) -> APTRepositoryFileError;
@@ -213,7 +119,8 @@ impl APTRepositoryFileImpl for APTRepositoryFile {
return Ok(None);
}
- let file_type = APTRepositoryFileType::try_from(&extension[..])
+ let file_type = extension[..]
+ .parse()
.map_err(|_| new_err("invalid extension"))?;
if !file_name
@@ -250,15 +157,15 @@ impl APTRepositoryFileImpl for APTRepositoryFile {
}
}
- fn read_with_digest(&self) -> Result<(Vec<u8>, [u8; 32]), APTRepositoryFileError> {
+ fn read_with_digest(&self) -> Result<(Vec<u8>, ConfigDigest), APTRepositoryFileError> {
if let Some(path) = &self.path {
let content = std::fs::read(path).map_err(|err| self.err(format_err!("{}", err)))?;
- let digest = openssl::sha::sha256(&content);
+ let digest = ConfigDigest::from_slice(&content);
Ok((content, digest))
} else if let Some(ref content) = self.content {
let content = content.as_bytes();
- let digest = openssl::sha::sha256(content);
+ let digest = ConfigDigest::from_slice(content);
Ok((content.to_vec(), digest))
} else {
Err(self.err(format_err!(
@@ -308,13 +215,13 @@ impl APTRepositoryFileImpl for APTRepositoryFile {
}
};
- if let Some(digest) = self.digest {
+ if let Some(digest) = &self.digest {
if !self.exists() {
return Err(self.err(format_err!("digest specified, but file does not exist")));
}
let (_, current_digest) = self.read_with_digest()?;
- if digest != current_digest {
+ if digest != ¤t_digest {
return Err(self.err(format_err!("digest mismatch")));
}
}
diff --git a/proxmox-apt/src/repositories/file/list_parser.rs b/proxmox-apt/src/repositories/file/list_parser.rs
index 93bbcc12..8681509a 100644
--- a/proxmox-apt/src/repositories/file/list_parser.rs
+++ b/proxmox-apt/src/repositories/file/list_parser.rs
@@ -184,7 +184,7 @@ impl<R: BufRead> APTListFileParser<R> {
// e.g. quoted "deb" is not accepted by APT, so no need for quote word parsing here
line = match line.split_once(|c| char::is_ascii_whitespace(&c)) {
Some((package_type, rest)) => {
- repo.types.push(package_type.try_into()?);
+ repo.types.push(package_type.parse()?);
rest
}
None => return Ok(None), // empty line
diff --git a/proxmox-apt/src/repositories/file/sources_parser.rs b/proxmox-apt/src/repositories/file/sources_parser.rs
index 213db9d6..017162bb 100644
--- a/proxmox-apt/src/repositories/file/sources_parser.rs
+++ b/proxmox-apt/src/repositories/file/sources_parser.rs
@@ -108,7 +108,7 @@ impl<R: BufRead> APTSourcesFileParser<R> {
}
let mut types = Vec::<APTRepositoryPackageType>::new();
for package_type in values {
- types.push((&package_type[..]).try_into()?);
+ types.push((&package_type[..]).parse()?);
}
repo.types = types;
}
diff --git a/proxmox-apt/src/repositories/mod.rs b/proxmox-apt/src/repositories/mod.rs
index 014f1820..7768a47a 100644
--- a/proxmox-apt/src/repositories/mod.rs
+++ b/proxmox-apt/src/repositories/mod.rs
@@ -4,21 +4,22 @@ use std::path::PathBuf;
use anyhow::{bail, Error};
mod repository;
-pub use repository::APTRepositoryImpl;
-pub use repository::{
- APTRepository, APTRepositoryFileType, APTRepositoryOption, APTRepositoryPackageType,
+use proxmox_apt_api_types::{
+ APTRepository, APTRepositoryFile, APTRepositoryFileError, APTRepositoryFileType,
+ APTRepositoryHandle, APTRepositoryInfo, APTRepositoryOption, APTRepositoryPackageType,
+ APTStandardRepository,
};
+use proxmox_config_digest::ConfigDigest;
+pub use repository::APTRepositoryImpl;
mod file;
pub use file::APTRepositoryFileImpl;
-pub use file::{APTRepositoryFile, APTRepositoryFileError, APTRepositoryInfo};
mod release;
pub use release::{get_current_release_codename, DebianCodename};
mod standard;
-pub use standard::APTRepositoryHandleImpl;
-pub use standard::{APTRepositoryHandle, APTStandardRepository};
+pub use standard::{APTRepositoryHandleImpl, APTStandardRepositoryImpl};
const APT_SOURCES_LIST_FILENAME: &str = "/etc/apt/sources.list";
const APT_SOURCES_LIST_DIRECTORY: &str = "/etc/apt/sources.list.d/";
@@ -28,7 +29,7 @@ const APT_SOURCES_LIST_DIRECTORY: &str = "/etc/apt/sources.list.d/";
/// The digest is invariant with respect to file order.
///
/// Files without a digest are ignored.
-fn common_digest(files: &[APTRepositoryFile]) -> [u8; 32] {
+fn common_digest(files: &[APTRepositoryFile]) -> ConfigDigest {
let mut digests = BTreeMap::new();
for file in files.iter() {
@@ -43,7 +44,7 @@ fn common_digest(files: &[APTRepositoryFile]) -> [u8; 32] {
}
}
- openssl::sha::sha256(&common_raw[..])
+ ConfigDigest::from_slice(&common_raw[..])
}
/// Provides additional information about the repositories.
@@ -86,22 +87,22 @@ pub fn standard_repositories(
suite: DebianCodename,
) -> Vec<APTStandardRepository> {
let mut result = vec![
- APTStandardRepository::from(APTRepositoryHandle::Enterprise),
- APTStandardRepository::from(APTRepositoryHandle::NoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::Test),
+ APTStandardRepository::from_handle(APTRepositoryHandle::Enterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::NoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::Test),
];
if product == "pve" {
result.append(&mut vec![
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyEnterprise),
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyNoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyTest),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyEnterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyNoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyTest),
]);
if suite == DebianCodename::Bookworm {
result.append(&mut vec![
- APTStandardRepository::from(APTRepositoryHandle::CephReefEnterprise),
- APTStandardRepository::from(APTRepositoryHandle::CephReefNoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::CephReefTest),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefEnterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefNoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefTest),
]);
}
}
@@ -128,7 +129,7 @@ pub fn standard_repositories(
pub type Repositories = (
Vec<APTRepositoryFile>,
Vec<APTRepositoryFileError>,
- [u8; 32],
+ ConfigDigest,
);
/// Returns all APT repositories configured in `/etc/apt/sources.list` and
diff --git a/proxmox-apt/src/repositories/repository.rs b/proxmox-apt/src/repositories/repository.rs
index a07db7cb..596c6385 100644
--- a/proxmox-apt/src/repositories/repository.rs
+++ b/proxmox-apt/src/repositories/repository.rs
@@ -1,193 +1,12 @@
-use std::fmt::Display;
use std::io::{BufRead, BufReader, Write};
use std::path::PathBuf;
use anyhow::{bail, format_err, Error};
-use serde::{Deserialize, Serialize};
-use proxmox_schema::api;
-
-use crate::repositories::standard::APTRepositoryHandle;
use crate::repositories::standard::APTRepositoryHandleImpl;
-
-#[api]
-#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
-#[serde(rename_all = "lowercase")]
-pub enum APTRepositoryFileType {
- /// One-line-style format
- List,
- /// DEB822-style format
- Sources,
-}
-
-impl TryFrom<&str> for APTRepositoryFileType {
- type Error = Error;
-
- fn try_from(file_type: &str) -> Result<Self, Error> {
- match file_type {
- "list" => Ok(APTRepositoryFileType::List),
- "sources" => Ok(APTRepositoryFileType::Sources),
- _ => bail!("invalid file type '{file_type}'"),
- }
- }
-}
-
-impl Display for APTRepositoryFileType {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- match self {
- APTRepositoryFileType::List => write!(f, "list"),
- APTRepositoryFileType::Sources => write!(f, "sources"),
- }
- }
-}
-
-#[api]
-#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
-#[serde(rename_all = "kebab-case")]
-pub enum APTRepositoryPackageType {
- /// Debian package
- Deb,
- /// Debian source package
- DebSrc,
-}
-
-impl TryFrom<&str> for APTRepositoryPackageType {
- type Error = Error;
-
- fn try_from(package_type: &str) -> Result<Self, Error> {
- match package_type {
- "deb" => Ok(APTRepositoryPackageType::Deb),
- "deb-src" => Ok(APTRepositoryPackageType::DebSrc),
- _ => bail!("invalid package type '{package_type}'"),
- }
- }
-}
-
-impl Display for APTRepositoryPackageType {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- match self {
- APTRepositoryPackageType::Deb => write!(f, "deb"),
- APTRepositoryPackageType::DebSrc => write!(f, "deb-src"),
- }
- }
-}
-
-#[api(
- properties: {
- Key: {
- description: "Option key.",
- type: String,
- },
- Values: {
- description: "Option values.",
- type: Array,
- items: {
- description: "Value.",
- type: String,
- },
- },
- },
-)]
-#[derive(Debug, Clone, Serialize, Deserialize)]
-#[serde(rename_all = "PascalCase")] // for consistency
-/// Additional options for an APT repository.
-/// Used for both single- and mutli-value options.
-pub struct APTRepositoryOption {
- /// Option key.
- pub key: String,
- /// Option value(s).
- pub values: Vec<String>,
-}
-
-#[api(
- properties: {
- Types: {
- description: "List of package types.",
- type: Array,
- items: {
- type: APTRepositoryPackageType,
- },
- },
- URIs: {
- description: "List of repository URIs.",
- type: Array,
- items: {
- description: "Repository URI.",
- type: String,
- },
- },
- Suites: {
- description: "List of distributions.",
- type: Array,
- items: {
- description: "Package distribution.",
- type: String,
- },
- },
- Components: {
- description: "List of repository components.",
- type: Array,
- items: {
- description: "Repository component.",
- type: String,
- },
- },
- Options: {
- type: Array,
- optional: true,
- items: {
- type: APTRepositoryOption,
- },
- },
- Comment: {
- description: "Associated comment.",
- type: String,
- optional: true,
- },
- FileType: {
- type: APTRepositoryFileType,
- },
- Enabled: {
- description: "Whether the repository is enabled or not.",
- type: Boolean,
- },
- },
-)]
-#[derive(Debug, Clone, Serialize, Deserialize)]
-#[serde(rename_all = "PascalCase")]
-/// Describes an APT repository.
-pub struct APTRepository {
- /// List of package types.
- #[serde(default, skip_serializing_if = "Vec::is_empty")]
- pub types: Vec<APTRepositoryPackageType>,
-
- /// List of repository URIs.
- #[serde(default, skip_serializing_if = "Vec::is_empty")]
- #[serde(rename = "URIs")]
- pub uris: Vec<String>,
-
- /// List of package distributions.
- #[serde(default, skip_serializing_if = "Vec::is_empty")]
- pub suites: Vec<String>,
-
- /// List of repository components.
- #[serde(default, skip_serializing_if = "Vec::is_empty")]
- pub components: Vec<String>,
-
- /// Additional options.
- #[serde(default, skip_serializing_if = "Vec::is_empty")]
- pub options: Vec<APTRepositoryOption>,
-
- /// Associated comment.
- #[serde(default, skip_serializing_if = "String::is_empty")]
- pub comment: String,
-
- /// Format of the defining file.
- pub file_type: APTRepositoryFileType,
-
- /// Whether the repository is enabled or not.
- pub enabled: bool,
-}
+use proxmox_apt_api_types::{
+ APTRepository, APTRepositoryFileType, APTRepositoryHandle, APTRepositoryOption,
+};
pub trait APTRepositoryImpl {
/// Crates an empty repository.
diff --git a/proxmox-apt/src/repositories/standard.rs b/proxmox-apt/src/repositories/standard.rs
index 7858fac4..64fdea2a 100644
--- a/proxmox-apt/src/repositories/standard.rs
+++ b/proxmox-apt/src/repositories/standard.rs
@@ -1,70 +1,14 @@
-use std::fmt::Display;
-
-use anyhow::{bail, Error};
-use serde::{Deserialize, Serialize};
-
-use crate::repositories::repository::{
- APTRepository, APTRepositoryFileType, APTRepositoryPackageType,
+use proxmox_apt_api_types::{
+ APTRepository, APTRepositoryFileType, APTRepositoryHandle, APTRepositoryPackageType,
+ APTStandardRepository,
};
-use proxmox_schema::api;
-
-#[api(
- properties: {
- handle: {
- description: "Handle referencing a standard repository.",
- type: String,
- },
- },
-)]
-#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
-#[serde(rename_all = "kebab-case")]
-/// Reference to a standard repository and configuration status.
-pub struct APTStandardRepository {
- /// Handle referencing a standard repository.
- pub handle: APTRepositoryHandle,
-
- /// Configuration status of the associated repository, where `None` means
- /// not configured, and `Some(bool)` indicates enabled or disabled.
- #[serde(skip_serializing_if = "Option::is_none")]
- pub status: Option<bool>,
-
- /// Display name of the repository.
- pub name: String,
-
- /// Description of the repository.
- pub description: String,
-}
-
-#[api]
-#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq)]
-#[serde(rename_all = "kebab-case")]
-/// Handles for Proxmox repositories.
-pub enum APTRepositoryHandle {
- /// The enterprise repository for production use.
- Enterprise,
- /// The repository that can be used without subscription.
- NoSubscription,
- /// The test repository.
- Test,
- /// Ceph Quincy enterprise repository.
- CephQuincyEnterprise,
- /// Ceph Quincy no-subscription repository.
- CephQuincyNoSubscription,
- /// Ceph Quincy test repository.
- CephQuincyTest,
- // TODO: Add separate enum for ceph releases and use something like
- // `CephTest(CephReleaseCodename),` once the API macro supports it.
- /// Ceph Reef enterprise repository.
- CephReefEnterprise,
- /// Ceph Reef no-subscription repository.
- CephReefNoSubscription,
- /// Ceph Reef test repository.
- CephReefTest,
+pub trait APTStandardRepositoryImpl {
+ fn from_handle(handle: APTRepositoryHandle) -> APTStandardRepository;
}
-impl From<APTRepositoryHandle> for APTStandardRepository {
- fn from(handle: APTRepositoryHandle) -> Self {
+impl APTStandardRepositoryImpl for APTStandardRepository {
+ fn from_handle(handle: APTRepositoryHandle) -> APTStandardRepository {
APTStandardRepository {
handle,
status: None,
@@ -74,43 +18,6 @@ impl From<APTRepositoryHandle> for APTStandardRepository {
}
}
-impl TryFrom<&str> for APTRepositoryHandle {
- type Error = Error;
-
- fn try_from(string: &str) -> Result<Self, Error> {
- match string {
- "enterprise" => Ok(APTRepositoryHandle::Enterprise),
- "no-subscription" => Ok(APTRepositoryHandle::NoSubscription),
- "test" => Ok(APTRepositoryHandle::Test),
- "ceph-quincy-enterprise" => Ok(APTRepositoryHandle::CephQuincyEnterprise),
- "ceph-quincy-no-subscription" => Ok(APTRepositoryHandle::CephQuincyNoSubscription),
- "ceph-quincy-test" => Ok(APTRepositoryHandle::CephQuincyTest),
- "ceph-reef-enterprise" => Ok(APTRepositoryHandle::CephReefEnterprise),
- "ceph-reef-no-subscription" => Ok(APTRepositoryHandle::CephReefNoSubscription),
- "ceph-reef-test" => Ok(APTRepositoryHandle::CephReefTest),
- _ => bail!("unknown repository handle '{}'", string),
- }
- }
-}
-
-impl Display for APTRepositoryHandle {
- fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
- match self {
- APTRepositoryHandle::Enterprise => write!(f, "enterprise"),
- APTRepositoryHandle::NoSubscription => write!(f, "no-subscription"),
- APTRepositoryHandle::Test => write!(f, "test"),
- APTRepositoryHandle::CephQuincyEnterprise => write!(f, "ceph-quincy-enterprise"),
- APTRepositoryHandle::CephQuincyNoSubscription => {
- write!(f, "ceph-quincy-no-subscription")
- }
- APTRepositoryHandle::CephQuincyTest => write!(f, "ceph-quincy-test"),
- APTRepositoryHandle::CephReefEnterprise => write!(f, "ceph-reef-enterprise"),
- APTRepositoryHandle::CephReefNoSubscription => write!(f, "ceph-reef-no-subscription"),
- APTRepositoryHandle::CephReefTest => write!(f, "ceph-reef-test"),
- }
- }
-}
-
pub trait APTRepositoryHandleImpl {
/// Get the description for the repository.
fn description(self) -> String;
diff --git a/proxmox-apt/tests/repositories.rs b/proxmox-apt/tests/repositories.rs
index 37d665bf..228ef696 100644
--- a/proxmox-apt/tests/repositories.rs
+++ b/proxmox-apt/tests/repositories.rs
@@ -9,7 +9,7 @@ use proxmox_apt::repositories::{
APTRepositoryHandle, APTRepositoryInfo, APTStandardRepository, DebianCodename,
};
use proxmox_apt::repositories::{
- APTRepositoryFileImpl, APTRepositoryHandleImpl, APTRepositoryImpl,
+ APTRepositoryFileImpl, APTRepositoryHandleImpl, APTRepositoryImpl, APTStandardRepositoryImpl,
};
fn create_clean_directory(path: &PathBuf) -> Result<(), Error> {
@@ -114,7 +114,7 @@ fn test_digest() -> Result<(), Error> {
let new_path = write_dir.join(path.file_name().unwrap());
file.path = Some(new_path.clone().into_os_string().into_string().unwrap());
- let old_digest = file.digest.unwrap();
+ let old_digest = file.digest.clone().unwrap();
// file does not exist yet...
assert!(file.read_with_digest().is_err());
@@ -132,7 +132,7 @@ fn test_digest() -> Result<(), Error> {
repo.enabled = !repo.enabled;
// ...then it should work
- file.digest = Some(old_digest);
+ file.digest = Some(old_digest.clone());
file.write()?;
// expect a different digest, because the repo was modified
@@ -361,15 +361,15 @@ fn test_standard_repositories() -> Result<(), Error> {
let read_dir = test_dir.join("sources.list.d");
let mut expected = vec![
- APTStandardRepository::from(APTRepositoryHandle::Enterprise),
- APTStandardRepository::from(APTRepositoryHandle::NoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::Test),
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyEnterprise),
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyNoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::CephQuincyTest),
- APTStandardRepository::from(APTRepositoryHandle::CephReefEnterprise),
- APTStandardRepository::from(APTRepositoryHandle::CephReefNoSubscription),
- APTStandardRepository::from(APTRepositoryHandle::CephReefTest),
+ APTStandardRepository::from_handle(APTRepositoryHandle::Enterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::NoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::Test),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyEnterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyNoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephQuincyTest),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefEnterprise),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefNoSubscription),
+ APTStandardRepository::from_handle(APTRepositoryHandle::CephReefTest),
];
let absolute_suite_list = read_dir.join("absolute_suite.list");
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox 5/6] apt: add cache feature
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
` (2 preceding siblings ...)
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 4/6] apt: use api types from apt-api-types crate Wolfgang Bumiller
@ 2024-07-09 6:18 ` Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config Wolfgang Bumiller
4 siblings, 0 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Save/read package state from a file, and add the api functions to manipulate
that state.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
proxmox-apt/Cargo.toml | 17 ++
proxmox-apt/debian/control | 23 +++
proxmox-apt/src/api.rs | 143 ++++++++++++++
proxmox-apt/src/cache.rs | 301 ++++++++++++++++++++++++++++++
proxmox-apt/src/cache_api.rs | 208 +++++++++++++++++++++
proxmox-apt/src/lib.rs | 9 +
proxmox-apt/tests/repositories.rs | 8 +-
7 files changed, 706 insertions(+), 3 deletions(-)
create mode 100644 proxmox-apt/src/api.rs
create mode 100644 proxmox-apt/src/cache.rs
create mode 100644 proxmox-apt/src/cache_api.rs
diff --git a/proxmox-apt/Cargo.toml b/proxmox-apt/Cargo.toml
index bbd4ff89..923be446 100644
--- a/proxmox-apt/Cargo.toml
+++ b/proxmox-apt/Cargo.toml
@@ -22,3 +22,20 @@ rfc822-like = "0.2.1"
proxmox-apt-api-types.workspace = true
proxmox-config-digest = { workspace = true, features = ["openssl"] }
+proxmox-sys.workspace = true
+
+apt-pkg-native = { version = "0.3.2", optional = true }
+regex = { workspace = true, optional = true }
+nix = { workspace = true, optional = true }
+log = { workspace = true, optional = true }
+proxmox-schema = { workspace = true, optional = true }
+
+[features]
+default = []
+cache = [
+ "dep:apt-pkg-native",
+ "dep:regex",
+ "dep:nix",
+ "dep:log",
+ "dep:proxmox-schema",
+]
diff --git a/proxmox-apt/debian/control b/proxmox-apt/debian/control
index 347631e6..7e0b79b1 100644
--- a/proxmox-apt/debian/control
+++ b/proxmox-apt/debian/control
@@ -13,6 +13,7 @@ Build-Depends: debhelper (>= 12),
librust-proxmox-apt-api-types-1+default-dev <!nocheck>,
librust-proxmox-config-digest-0.1+default-dev <!nocheck>,
librust-proxmox-config-digest-0.1+openssl-dev <!nocheck>,
+ librust-proxmox-sys-0.5+default-dev (>= 0.5.7-~~) <!nocheck>,
librust-rfc822-like-0.2+default-dev (>= 0.2.1-~~) <!nocheck>,
librust-serde-1+default-dev <!nocheck>,
librust-serde-1+derive-dev <!nocheck>,
@@ -37,10 +38,13 @@ Depends:
librust-proxmox-apt-api-types-1+default-dev,
librust-proxmox-config-digest-0.1+default-dev,
librust-proxmox-config-digest-0.1+openssl-dev,
+ librust-proxmox-sys-0.5+default-dev (>= 0.5.7-~~),
librust-rfc822-like-0.2+default-dev (>= 0.2.1-~~),
librust-serde-1+default-dev,
librust-serde-1+derive-dev,
librust-serde-json-1+default-dev
+Suggests:
+ librust-proxmox-apt+cache-dev (= ${binary:Version})
Provides:
librust-proxmox-apt+default-dev (= ${binary:Version}),
librust-proxmox-apt-0-dev (= ${binary:Version}),
@@ -51,3 +55,22 @@ Provides:
librust-proxmox-apt-0.10.10+default-dev (= ${binary:Version})
Description: Proxmox library for APT - Rust source code
Source code for Debianized Rust crate "proxmox-apt"
+
+Package: librust-proxmox-apt+cache-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-proxmox-apt-dev (= ${binary:Version}),
+ librust-apt-pkg-native-0.3+default-dev (>= 0.3.2-~~),
+ librust-log-0.4+default-dev (>= 0.4.17-~~),
+ librust-nix-0.26+default-dev (>= 0.26.1-~~),
+ librust-proxmox-schema-3+default-dev (>= 3.1.1-~~),
+ librust-regex-1+default-dev (>= 1.5-~~)
+Provides:
+ librust-proxmox-apt-0+cache-dev (= ${binary:Version}),
+ librust-proxmox-apt-0.10+cache-dev (= ${binary:Version}),
+ librust-proxmox-apt-0.10.10+cache-dev (= ${binary:Version})
+Description: Proxmox library for APT - feature "cache"
+ This metapackage enables feature "cache" for the Rust proxmox-apt crate, by
+ pulling in any additional dependencies needed by that feature.
diff --git a/proxmox-apt/src/api.rs b/proxmox-apt/src/api.rs
new file mode 100644
index 00000000..af01048e
--- /dev/null
+++ b/proxmox-apt/src/api.rs
@@ -0,0 +1,143 @@
+// API function that work without feature "cache"
+
+use anyhow::{bail, Error};
+
+use proxmox_apt_api_types::{
+ APTChangeRepositoryOptions, APTGetChangelogOptions, APTRepositoriesResult, APTRepositoryFile,
+ APTRepositoryHandle,
+};
+use proxmox_config_digest::ConfigDigest;
+
+use crate::repositories::{APTRepositoryFileImpl, APTRepositoryImpl};
+
+/// Retrieve the changelog of the specified package.
+pub fn get_changelog(options: &APTGetChangelogOptions) -> Result<String, Error> {
+ let mut command = std::process::Command::new("apt-get");
+ command.arg("changelog");
+ command.arg("-qq"); // don't display download progress
+ if let Some(ver) = &options.version {
+ command.arg(format!("{}={}", options.name, ver));
+ } else {
+ command.arg(&options.name);
+ }
+ let output = proxmox_sys::command::run_command(command, None)?;
+
+ Ok(output)
+}
+
+/// Get APT repository information.
+pub fn list_repositories(product: &str) -> Result<APTRepositoriesResult, Error> {
+ let (files, errors, digest) = crate::repositories::repositories()?;
+
+ let suite = crate::repositories::get_current_release_codename()?;
+
+ let infos = crate::repositories::check_repositories(&files, suite);
+ let standard_repos = crate::repositories::standard_repositories(&files, product, suite);
+
+ Ok(APTRepositoriesResult {
+ files,
+ errors,
+ digest,
+ infos,
+ standard_repos,
+ })
+}
+
+/// Add the repository identified by the `handle`.
+/// If the repository is already configured, it will be set to enabled.
+///
+/// The `digest` parameter asserts that the configuration has not been modified.
+pub fn add_repository_handle(
+ product: &str,
+ handle: APTRepositoryHandle,
+ digest: Option<ConfigDigest>,
+) -> Result<(), Error> {
+ let (mut files, errors, current_digest) = crate::repositories::repositories()?;
+
+ current_digest.detect_modification(digest.as_ref())?;
+
+ let suite = crate::repositories::get_current_release_codename()?;
+
+ // check if it's already configured first
+ for file in files.iter_mut() {
+ for repo in file.repositories.iter_mut() {
+ if repo.is_referenced_repository(handle, "pbs", &suite.to_string()) {
+ if repo.enabled {
+ return Ok(());
+ }
+
+ repo.set_enabled(true);
+ file.write()?;
+
+ return Ok(());
+ }
+ }
+ }
+
+ let (repo, path) = crate::repositories::get_standard_repository(handle, product, suite);
+
+ if let Some(error) = errors.iter().find(|error| error.path == path) {
+ bail!(
+ "unable to parse existing file {} - {}",
+ error.path,
+ error.error,
+ );
+ }
+
+ if let Some(file) = files
+ .iter_mut()
+ .find(|file| file.path.as_ref() == Some(&path))
+ {
+ file.repositories.push(repo);
+
+ file.write()?;
+ } else {
+ let mut file = match APTRepositoryFile::new(&path)? {
+ Some(file) => file,
+ None => bail!("invalid path - {}", path),
+ };
+
+ file.repositories.push(repo);
+
+ file.write()?;
+ }
+
+ Ok(())
+}
+
+/// Change the properties of the specified repository.
+///
+/// The `digest` parameter asserts that the configuration has not been modified.
+pub fn change_repository(
+ path: &str,
+ index: usize,
+ options: &APTChangeRepositoryOptions,
+ digest: Option<ConfigDigest>,
+) -> Result<(), Error> {
+ let (mut files, errors, current_digest) = crate::repositories::repositories()?;
+
+ current_digest.detect_modification(digest.as_ref())?;
+
+ if let Some(error) = errors.iter().find(|error| error.path == path) {
+ bail!("unable to parse file {} - {}", error.path, error.error);
+ }
+
+ if let Some(file) = files
+ .iter_mut()
+ .find(|file| file.path.as_deref() == Some(path))
+ {
+ if let Some(repo) = file.repositories.get_mut(index) {
+ if let Some(enabled) = options.enabled {
+ repo.set_enabled(enabled);
+ }
+
+ file.write()?;
+ } else {
+ bail!("invalid index - {}", index);
+ }
+ } else {
+ bail!("invalid path - {}", path);
+ }
+
+ Ok(())
+}
diff --git a/proxmox-apt/src/cache.rs b/proxmox-apt/src/cache.rs
new file mode 100644
index 00000000..03315013
--- /dev/null
+++ b/proxmox-apt/src/cache.rs
@@ -0,0 +1,301 @@
+use std::collections::HashMap;
+use std::collections::HashSet;
+use std::path::Path;
+
+use anyhow::{bail, format_err, Error};
+use apt_pkg_native::Cache;
+
+use proxmox_schema::const_regex;
+use proxmox_sys::fs::{file_read_optional_string, replace_file, CreateOptions};
+
+use proxmox_apt_api_types::APTUpdateInfo;
+
+#[derive(Debug, serde::Serialize, serde::Deserialize)]
+/// Some information we cache about the package (update) state, like what pending update version
+/// we already notfied an user about
+pub struct PkgState {
+ /// simple map from package name to most recently notified (emailed) version
+ pub notified: Option<HashMap<String, String>>,
+ /// A list of pending updates
+ pub package_status: Vec<APTUpdateInfo>,
+}
+
+pub fn write_pkg_cache<P: AsRef<Path>>(apt_state_file: P, state: &PkgState) -> Result<(), Error> {
+ let serialized_state = serde_json::to_string(state)?;
+
+ replace_file(
+ apt_state_file,
+ serialized_state.as_bytes(),
+ CreateOptions::new(),
+ false,
+ )
+ .map_err(|err| format_err!("Error writing package cache - {}", err))?;
+ Ok(())
+}
+
+pub fn read_pkg_state<P: AsRef<Path>>(apt_state_file: P) -> Result<Option<PkgState>, Error> {
+ let serialized_state = match file_read_optional_string(apt_state_file) {
+ Ok(Some(raw)) => raw,
+ Ok(None) => return Ok(None),
+ Err(err) => bail!("could not read cached package state file - {}", err),
+ };
+
+ serde_json::from_str(&serialized_state)
+ .map(Some)
+ .map_err(|err| format_err!("could not parse cached package status - {}", err))
+}
+
+pub fn pkg_cache_expired<P: AsRef<Path>>(apt_state_file: P) -> Result<bool, Error> {
+ if let Ok(pbs_cache) = std::fs::metadata(apt_state_file) {
+ let apt_pkgcache = std::fs::metadata("/var/cache/apt/pkgcache.bin")?;
+ let dpkg_status = std::fs::metadata("/var/lib/dpkg/status")?;
+
+ let mtime = pbs_cache.modified()?;
+
+ if apt_pkgcache.modified()? <= mtime && dpkg_status.modified()? <= mtime {
+ return Ok(false);
+ }
+ }
+ Ok(true)
+}
+
+pub fn update_cache<P: AsRef<Path>>(apt_state_file: P) -> Result<PkgState, Error> {
+ let apt_state_file = apt_state_file.as_ref();
+ // update our cache
+ let all_upgradeable = list_installed_apt_packages(
+ |data| {
+ data.candidate_version == data.active_version
+ && data.installed_version != Some(data.candidate_version)
+ },
+ None,
+ );
+
+ let cache = match read_pkg_state(apt_state_file) {
+ Ok(Some(mut cache)) => {
+ cache.package_status = all_upgradeable;
+ cache
+ }
+ _ => PkgState {
+ notified: None,
+ package_status: all_upgradeable,
+ },
+ };
+ write_pkg_cache(apt_state_file, &cache)?;
+ Ok(cache)
+}
+
+const_regex! {
+ VERSION_EPOCH_REGEX = r"^\d+:";
+ FILENAME_EXTRACT_REGEX = r"^.*/.*?_(.*)_Packages$";
+}
+
+pub struct FilterData<'a> {
+ /// package name
+ pub package: &'a str,
+ /// this is version info returned by APT
+ pub installed_version: Option<&'a str>,
+ pub candidate_version: &'a str,
+
+ /// this is the version info the filter is supposed to check
+ pub active_version: &'a str,
+}
+
+enum PackagePreSelect {
+ OnlyInstalled,
+ OnlyNew,
+ All,
+}
+
+pub fn list_installed_apt_packages<F: Fn(FilterData) -> bool>(
+ filter: F,
+ only_versions_for: Option<&str>,
+) -> Vec<APTUpdateInfo> {
+ let mut ret = Vec::new();
+ let mut depends = HashSet::new();
+
+ // note: this is not an 'apt update', it just re-reads the cache from disk
+ let mut cache = Cache::get_singleton();
+ cache.reload();
+
+ let mut cache_iter = match only_versions_for {
+ Some(name) => cache.find_by_name(name),
+ None => cache.iter(),
+ };
+
+ loop {
+ match cache_iter.next() {
+ Some(view) => {
+ let di = if only_versions_for.is_some() {
+ query_detailed_info(PackagePreSelect::All, &filter, view, None)
+ } else {
+ query_detailed_info(
+ PackagePreSelect::OnlyInstalled,
+ &filter,
+ view,
+ Some(&mut depends),
+ )
+ };
+ if let Some(info) = di {
+ ret.push(info);
+ }
+
+ if only_versions_for.is_some() {
+ break;
+ }
+ }
+ None => {
+ drop(cache_iter);
+ // also loop through missing dependencies, as they would be installed
+ for pkg in depends.iter() {
+ let mut iter = cache.find_by_name(pkg);
+ let view = match iter.next() {
+ Some(view) => view,
+ None => continue, // package not found, ignore
+ };
+
+ let di = query_detailed_info(PackagePreSelect::OnlyNew, &filter, view, None);
+ if let Some(info) = di {
+ ret.push(info);
+ }
+ }
+ break;
+ }
+ }
+ }
+
+ ret
+}
+
+fn query_detailed_info<'a, F, V>(
+ pre_select: PackagePreSelect,
+ filter: F,
+ view: V,
+ depends: Option<&mut HashSet<String>>,
+) -> Option<APTUpdateInfo>
+where
+ F: Fn(FilterData) -> bool,
+ V: std::ops::Deref<Target = apt_pkg_native::sane::PkgView<'a>>,
+{
+ let current_version = view.current_version();
+ let candidate_version = view.candidate_version();
+
+ let (current_version, candidate_version) = match pre_select {
+ PackagePreSelect::OnlyInstalled => match (current_version, candidate_version) {
+ (Some(cur), Some(can)) => (Some(cur), can), // package installed and there is an update
+ (Some(cur), None) => (Some(cur.clone()), cur), // package installed and up-to-date
+ (None, Some(_)) => return None, // package could be installed
+ (None, None) => return None, // broken
+ },
+ PackagePreSelect::OnlyNew => match (current_version, candidate_version) {
+ (Some(_), Some(_)) => return None,
+ (Some(_), None) => return None,
+ (None, Some(can)) => (None, can),
+ (None, None) => return None,
+ },
+ PackagePreSelect::All => match (current_version, candidate_version) {
+ (Some(cur), Some(can)) => (Some(cur), can),
+ (Some(cur), None) => (Some(cur.clone()), cur),
+ (None, Some(can)) => (None, can),
+ (None, None) => return None,
+ },
+ };
+
+ // get additional information via nested APT 'iterators'
+ let mut view_iter = view.versions();
+ while let Some(ver) = view_iter.next() {
+ let package = view.name();
+ let version = ver.version();
+ let mut origin_res = "unknown".to_owned();
+ let mut section_res = "unknown".to_owned();
+ let mut priority_res = "unknown".to_owned();
+ let mut short_desc = package.clone();
+ let mut long_desc = "".to_owned();
+
+ let fd = FilterData {
+ package: package.as_str(),
+ installed_version: current_version.as_deref(),
+ candidate_version: &candidate_version,
+ active_version: &version,
+ };
+
+ if filter(fd) {
+ if let Some(section) = ver.section() {
+ section_res = section;
+ }
+
+ if let Some(prio) = ver.priority_type() {
+ priority_res = prio;
+ }
+
+ // assume every package has only one origin file (not
+ // origin, but origin *file*, for some reason those seem to
+ // be different concepts in APT)
+ let mut origin_iter = ver.origin_iter();
+ let origin = origin_iter.next();
+ if let Some(origin) = origin {
+ if let Some(sd) = origin.short_desc() {
+ short_desc = sd;
+ }
+
+ if let Some(ld) = origin.long_desc() {
+ long_desc = ld;
+ }
+
+ // the package files appear in priority order, meaning
+ // the one for the candidate version is first - this is fine
+ // however, as the source package should be the same for all
+ // versions anyway
+ let mut pkg_iter = origin.file();
+ let pkg_file = pkg_iter.next();
+ if let Some(pkg_file) = pkg_file {
+ if let Some(origin_name) = pkg_file.origin() {
+ origin_res = origin_name;
+ }
+ }
+ }
+
+ if let Some(depends) = depends {
+ let mut dep_iter = ver.dep_iter();
+ loop {
+ let dep = match dep_iter.next() {
+ Some(dep) if dep.dep_type() != "Depends" => continue,
+ Some(dep) => dep,
+ None => break,
+ };
+
+ let dep_pkg = dep.target_pkg();
+ let name = dep_pkg.name();
+
+ depends.insert(name);
+ }
+ }
+
+ return Some(APTUpdateInfo {
+ package,
+ title: short_desc,
+ arch: view.arch(),
+ description: long_desc,
+ origin: origin_res,
+ version: candidate_version.clone(),
+ old_version: match current_version {
+ Some(vers) => vers,
+ None => "".to_owned(),
+ },
+ priority: priority_res,
+ section: section_res,
+ extra_info: None,
+ });
+ }
+ }
+
+ None
+}
+
+pub fn sort_package_list(packages: &mut Vec<APTUpdateInfo>) {
+ let cache = apt_pkg_native::Cache::get_singleton();
+ packages.sort_by(|left, right| {
+ cache
+ .compare_versions(&left.old_version, &right.old_version)
+ .reverse()
+ });
+}
diff --git a/proxmox-apt/src/cache_api.rs b/proxmox-apt/src/cache_api.rs
new file mode 100644
index 00000000..979f47ed
--- /dev/null
+++ b/proxmox-apt/src/cache_api.rs
@@ -0,0 +1,208 @@
+// API function that need feature "cache"
+use std::path::Path;
+
+use anyhow::{bail, format_err, Error};
+use std::os::unix::prelude::OsStrExt;
+
+use proxmox_apt_api_types::{APTUpdateInfo, APTUpdateOptions};
+
+/// List available APT updates
+///
+/// Automatically updates an expired package cache.
+pub fn list_available_apt_update<P: AsRef<Path>>(
+ apt_state_file: P,
+) -> Result<Vec<APTUpdateInfo>, Error> {
+ let apt_state_file = apt_state_file.as_ref();
+ if let Ok(false) = crate::cache::pkg_cache_expired(apt_state_file) {
+ if let Ok(Some(cache)) = crate::cache::read_pkg_state(apt_state_file) {
+ return Ok(cache.package_status);
+ }
+ }
+
+ let cache = crate::cache::update_cache(apt_state_file)?;
+
+ Ok(cache.package_status)
+}
+
+/// Update the APT database
+///
+/// You should update the APT proxy configuration before running this.
+pub fn update_database<P: AsRef<Path>>(
+ apt_state_file: P,
+ options: &APTUpdateOptions,
+ send_updates_available: impl Fn(&[&APTUpdateInfo]) -> Result<(), Error>,
+) -> Result<(), Error> {
+ let apt_state_file = apt_state_file.as_ref();
+
+ let quiet = options.quiet.unwrap_or(false);
+ let notify = options.notify.unwrap_or(false);
+
+ if !quiet {
+ log::info!("starting apt-get update")
+ }
+
+ let mut command = std::process::Command::new("apt-get");
+ command.arg("update");
+
+ // apt "errors" quite easily, and run_command is a bit rigid, so handle this inline for now.
+ let output = command
+ .output()
+ .map_err(|err| format_err!("failed to execute {:?} - {}", command, err))?;
+
+ if !quiet {
+ log::info!("{}", String::from_utf8(output.stdout)?);
+ }
+
+ // TODO: improve run_command to allow outputting both, stderr and stdout
+ if !output.status.success() {
+ if output.status.code().is_some() {
+ let msg = String::from_utf8(output.stderr)
+ .map(|m| {
+ if m.is_empty() {
+ String::from("no error message")
+ } else {
+ m
+ }
+ })
+ .unwrap_or_else(|_| String::from("non utf8 error message (suppressed)"));
+ log::warn!("{msg}");
+ } else {
+ bail!("terminated by signal");
+ }
+ }
+
+ let mut cache = crate::cache::update_cache(apt_state_file)?;
+
+ if notify {
+ let mut notified = match cache.notified {
+ Some(notified) => notified,
+ None => std::collections::HashMap::new(),
+ };
+ let mut to_notify: Vec<&APTUpdateInfo> = Vec::new();
+
+ for pkg in &cache.package_status {
+ match notified.insert(pkg.package.to_owned(), pkg.version.to_owned()) {
+ Some(notified_version) => {
+ if notified_version != pkg.version {
+ to_notify.push(pkg);
+ }
+ }
+ None => to_notify.push(pkg),
+ }
+ }
+ if !to_notify.is_empty() {
+ to_notify.sort_unstable_by_key(|k| &k.package);
+ send_updates_available(&to_notify)?;
+ }
+ cache.notified = Some(notified);
+ crate::cache::write_pkg_cache(apt_state_file, &cache)?;
+ }
+
+ Ok(())
+}
+
+/// Get package information for a list of important product packages.
+///
+/// We first list the product virtual package (i.e. `proxmox-backup`), with extra
+/// information about the running kernel.
+///
+/// Next is the api_server_package, with extra information abnout the running api
+/// server version.
+///
+/// The list of installed kernel packages follows.
+///
+/// We the add an entry for all packages in package_list, even if they are
+/// not installed.
+pub fn get_package_versions(
+ product_virtual_package: &str,
+ api_server_package: &str,
+ running_api_server_version: &str,
+ package_list: &[&str],
+) -> Result<Vec<APTUpdateInfo>, Error> {
+ fn unknown_package(package: String, extra_info: Option<String>) -> APTUpdateInfo {
+ APTUpdateInfo {
+ package,
+ title: "unknown".into(),
+ arch: "unknown".into(),
+ description: "unknown".into(),
+ version: "unknown".into(),
+ old_version: "unknown".into(),
+ origin: "unknown".into(),
+ priority: "unknown".into(),
+ section: "unknown".into(),
+ extra_info,
+ }
+ }
+
+ let mut packages: Vec<APTUpdateInfo> = Vec::new();
+
+ let is_kernel =
+ |name: &str| name.starts_with("pve-kernel-") || name.starts_with("proxmox-kernel");
+
+ let installed_packages = crate::cache::list_installed_apt_packages(
+ |filter| {
+ filter.installed_version == Some(filter.active_version)
+ && (is_kernel(filter.package)
+ || (filter.package == product_virtual_package)
+ || (filter.package == api_server_package)
+ || package_list.contains(&filter.package))
+ },
+ None,
+ );
+
+ let running_kernel = format!(
+ "running kernel: {}",
+ std::str::from_utf8(nix::sys::utsname::uname()?.release().as_bytes())?.to_owned()
+ );
+
+ if let Some(product_virtual_package_info) = installed_packages
+ .iter()
+ .find(|pkg| pkg.package == product_virtual_package)
+ {
+ let mut product_virtual_package_info = product_virtual_package_info.clone();
+ product_virtual_package_info.extra_info = Some(running_kernel);
+ packages.push(product_virtual_package_info);
+ } else {
+ packages.push(unknown_package(
+ product_virtual_package.into(),
+ Some(running_kernel),
+ ));
+ }
+
+ if let Some(api_server_package_info) = installed_packages
+ .iter()
+ .find(|pkg| pkg.package == api_server_package)
+ {
+ let mut api_server_package_info = api_server_package_info.clone();
+ api_server_package_info.extra_info = Some(running_api_server_version.into());
+ packages.push(api_server_package_info);
+ } else {
+ packages.push(unknown_package(
+ api_server_package.into(),
+ Some(running_api_server_version.into()),
+ ));
+ }
+
+ let mut kernel_pkgs: Vec<APTUpdateInfo> = installed_packages
+ .iter()
+ .filter(|pkg| is_kernel(&pkg.package))
+ .cloned()
+ .collect();
+
+ crate::cache::sort_package_list(&mut kernel_pkgs);
+
+ packages.append(&mut kernel_pkgs);
+
+ // add entry for all packages we're interested in, even if not installed
+ for pkg in package_list.iter() {
+ if *pkg == product_virtual_package || *pkg == api_server_package {
+ continue;
+ }
+ match installed_packages.iter().find(|item| &item.package == pkg) {
+ Some(apt_pkg) => packages.push(apt_pkg.to_owned()),
+ None => packages.push(unknown_package(pkg.to_string(), None)),
+ }
+ }
+
+ Ok(packages)
+}
diff --git a/proxmox-apt/src/lib.rs b/proxmox-apt/src/lib.rs
index 60bf1d2a..f25ac90b 100644
--- a/proxmox-apt/src/lib.rs
+++ b/proxmox-apt/src/lib.rs
@@ -1,5 +1,14 @@
#![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))]
+mod api;
+pub use api::{add_repository_handle, change_repository, get_changelog, list_repositories};
+
+#[cfg(feature = "cache")]
+pub mod cache;
+#[cfg(feature = "cache")]
+mod cache_api;
+#[cfg(feature = "cache")]
+pub use cache_api::{get_package_versions, list_available_apt_update, update_database};
pub mod config;
pub mod deb822;
pub mod repositories;
diff --git a/proxmox-apt/tests/repositories.rs b/proxmox-apt/tests/repositories.rs
index 228ef696..e4a94525 100644
--- a/proxmox-apt/tests/repositories.rs
+++ b/proxmox-apt/tests/repositories.rs
@@ -5,11 +5,13 @@ use anyhow::{bail, format_err, Error};
use proxmox_apt::config::APTConfig;
use proxmox_apt::repositories::{
- check_repositories, get_current_release_codename, standard_repositories, APTRepositoryFile,
- APTRepositoryHandle, APTRepositoryInfo, APTStandardRepository, DebianCodename,
+ check_repositories, get_current_release_codename, standard_repositories, DebianCodename,
};
use proxmox_apt::repositories::{
- APTRepositoryFileImpl, APTRepositoryHandleImpl, APTRepositoryImpl, APTStandardRepositoryImpl,
+ APTRepositoryFileImpl, APTRepositoryImpl, APTStandardRepositoryImpl,
+};
+use proxmox_apt_api_types::{
+ APTRepositoryFile, APTRepositoryHandle, APTRepositoryInfo, APTStandardRepository,
};
fn create_clean_directory(path: &PathBuf) -> Result<(), Error> {
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
` (3 preceding siblings ...)
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 5/6] apt: add cache feature Wolfgang Bumiller
@ 2024-07-09 6:18 ` Wolfgang Bumiller
2024-07-09 10:34 ` Fiona Ebner
4 siblings, 1 reply; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 6:18 UTC (permalink / raw)
To: pbs-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Because it was only used for the test setup. Instead, we simply
add an apt_lists_dir parameter where we need it.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
proxmox-apt/src/api.rs | 5 +++-
proxmox-apt/src/config.rs | 35 ----------------------
proxmox-apt/src/lib.rs | 2 +-
proxmox-apt/src/repositories/file.rs | 6 ++--
proxmox-apt/src/repositories/mod.rs | 5 ++--
proxmox-apt/src/repositories/repository.rs | 15 +++++-----
proxmox-apt/tests/repositories.rs | 24 +++++----------
7 files changed, 25 insertions(+), 67 deletions(-)
delete mode 100644 proxmox-apt/src/config.rs
diff --git a/proxmox-apt/src/api.rs b/proxmox-apt/src/api.rs
index af01048e..14dfb53b 100644
--- a/proxmox-apt/src/api.rs
+++ b/proxmox-apt/src/api.rs
@@ -1,4 +1,5 @@
// API function that work without feature "cache"
+use std::path::Path;
use anyhow::{bail, Error};
@@ -27,11 +28,13 @@ pub fn get_changelog(options: &APTGetChangelogOptions) -> Result<String, Error>
/// Get APT repository information.
pub fn list_repositories(product: &str) -> Result<APTRepositoriesResult, Error> {
+ let apt_lists_dir = Path::new("/var/lib/apt/lists");
+
let (files, errors, digest) = crate::repositories::repositories()?;
let suite = crate::repositories::get_current_release_codename()?;
- let infos = crate::repositories::check_repositories(&files, suite);
+ let infos = crate::repositories::check_repositories(&files, suite, apt_lists_dir);
let standard_repos = crate::repositories::standard_repositories(&files, product, suite);
Ok(APTRepositoriesResult {
diff --git a/proxmox-apt/src/config.rs b/proxmox-apt/src/config.rs
deleted file mode 100644
index fcb66cbb..00000000
--- a/proxmox-apt/src/config.rs
+++ /dev/null
@@ -1,35 +0,0 @@
-use once_cell::sync::OnceCell;
-
-static GLOBAL_CONFIG: OnceCell<APTConfig> = OnceCell::new();
-
-/// APT configuration variables.
-pub struct APTConfig {
- /// Dir::State
- pub dir_state: String,
- /// Dir::State::Lists
- pub dir_state_lists: String,
-}
-
-impl APTConfig {
- /// Create a new configuration overriding the provided values.
- pub fn new(dir_state: Option<&str>, dir_state_lists: Option<&str>) -> Self {
- Self {
- dir_state: dir_state.unwrap_or("/var/lib/apt/").to_string(),
- dir_state_lists: dir_state_lists.unwrap_or("lists/").to_string(),
- }
- }
-}
-
-/// Get the configuration.
-///
-/// Initializes with default values if init() wasn't called before.
-pub fn get() -> &'static APTConfig {
- GLOBAL_CONFIG.get_or_init(|| APTConfig::new(None, None))
-}
-
-/// Initialize the configuration.
-///
-/// Only has an effect if no init() or get() has been called yet.
-pub fn init(config: APTConfig) -> &'static APTConfig {
- GLOBAL_CONFIG.get_or_init(|| config)
-}
diff --git a/proxmox-apt/src/lib.rs b/proxmox-apt/src/lib.rs
index f25ac90b..cd504739 100644
--- a/proxmox-apt/src/lib.rs
+++ b/proxmox-apt/src/lib.rs
@@ -9,6 +9,6 @@ pub mod cache;
mod cache_api;
#[cfg(feature = "cache")]
pub use cache_api::{get_package_versions, list_available_apt_update, update_database};
-pub mod config;
+
pub mod deb822;
pub mod repositories;
diff --git a/proxmox-apt/src/repositories/file.rs b/proxmox-apt/src/repositories/file.rs
index 21f612ef..078f6407 100644
--- a/proxmox-apt/src/repositories/file.rs
+++ b/proxmox-apt/src/repositories/file.rs
@@ -59,7 +59,7 @@ pub trait APTRepositoryFileImpl {
fn check_suites(&self, current_codename: DebianCodename) -> Vec<APTRepositoryInfo>;
/// Checks for official URIs.
- fn check_uris(&self) -> Vec<APTRepositoryInfo>;
+ fn check_uris(&self, apt_lists_dir: &Path) -> Vec<APTRepositoryInfo>;
}
impl APTRepositoryFileImpl for APTRepositoryFile {
@@ -357,7 +357,7 @@ impl APTRepositoryFileImpl for APTRepositoryFile {
infos
}
- fn check_uris(&self) -> Vec<APTRepositoryInfo> {
+ fn check_uris(&self, apt_lists_dir: &Path) -> Vec<APTRepositoryInfo> {
let mut infos = vec![];
let path = match &self.path {
@@ -366,7 +366,7 @@ impl APTRepositoryFileImpl for APTRepositoryFile {
};
for (n, repo) in self.repositories.iter().enumerate() {
- let mut origin = match repo.get_cached_origin() {
+ let mut origin = match repo.get_cached_origin(apt_lists_dir) {
Ok(option) => option,
Err(_) => None,
};
diff --git a/proxmox-apt/src/repositories/mod.rs b/proxmox-apt/src/repositories/mod.rs
index 7768a47a..451b356b 100644
--- a/proxmox-apt/src/repositories/mod.rs
+++ b/proxmox-apt/src/repositories/mod.rs
@@ -1,5 +1,5 @@
use std::collections::BTreeMap;
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
use anyhow::{bail, Error};
@@ -56,12 +56,13 @@ fn common_digest(files: &[APTRepositoryFile]) -> ConfigDigest {
pub fn check_repositories(
files: &[APTRepositoryFile],
current_suite: DebianCodename,
+ apt_lists_dir: &Path,
) -> Vec<APTRepositoryInfo> {
let mut infos = vec![];
for file in files.iter() {
infos.append(&mut file.check_suites(current_suite));
- infos.append(&mut file.check_uris());
+ infos.append(&mut file.check_uris(apt_lists_dir));
}
infos
diff --git a/proxmox-apt/src/repositories/repository.rs b/proxmox-apt/src/repositories/repository.rs
index 596c6385..62ad49d8 100644
--- a/proxmox-apt/src/repositories/repository.rs
+++ b/proxmox-apt/src/repositories/repository.rs
@@ -1,5 +1,5 @@
use std::io::{BufRead, BufReader, Write};
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
use anyhow::{bail, format_err, Error};
@@ -33,7 +33,7 @@ pub trait APTRepositoryImpl {
fn origin_from_uris(&self) -> Option<String>;
/// Get the `Origin:` value from a cached InRelease file.
- fn get_cached_origin(&self) -> Result<Option<String>, Error>;
+ fn get_cached_origin(&self, apt_lists_dir: &Path) -> Result<Option<String>, Error>;
/// Writes a repository in the corresponding format followed by a blank.
///
@@ -163,13 +163,13 @@ impl APTRepositoryImpl for APTRepository {
None
}
- fn get_cached_origin(&self) -> Result<Option<String>, Error> {
+ fn get_cached_origin(&self, apt_lists_dir: &Path) -> Result<Option<String>, Error> {
for uri in self.uris.iter() {
for suite in self.suites.iter() {
- let mut file = release_filename(uri, suite, false);
+ let mut file = release_filename(apt_lists_dir, uri, suite, false);
if !file.exists() {
- file = release_filename(uri, suite, true);
+ file = release_filename(apt_lists_dir, uri, suite, true);
if !file.exists() {
continue;
}
@@ -206,9 +206,8 @@ impl APTRepositoryImpl for APTRepository {
}
/// Get the path to the cached (In)Release file.
-fn release_filename(uri: &str, suite: &str, detached: bool) -> PathBuf {
- let mut path = PathBuf::from(&crate::config::get().dir_state);
- path.push(&crate::config::get().dir_state_lists);
+fn release_filename(apt_lists_dir: &Path, uri: &str, suite: &str, detached: bool) -> PathBuf {
+ let mut path = PathBuf::from(apt_lists_dir);
let encoded_uri = uri_to_filename(uri);
let filename = if detached { "Release" } else { "InRelease" };
diff --git a/proxmox-apt/tests/repositories.rs b/proxmox-apt/tests/repositories.rs
index e4a94525..e4efcab6 100644
--- a/proxmox-apt/tests/repositories.rs
+++ b/proxmox-apt/tests/repositories.rs
@@ -2,8 +2,6 @@ use std::path::PathBuf;
use anyhow::{bail, format_err, Error};
-use proxmox_apt::config::APTConfig;
-
use proxmox_apt::repositories::{
check_repositories, get_current_release_codename, standard_repositories, DebianCodename,
};
@@ -185,17 +183,13 @@ fn test_empty_write() -> Result<(), Error> {
fn test_check_repositories() -> Result<(), Error> {
let test_dir = std::env::current_dir()?.join("tests");
let read_dir = test_dir.join("sources.list.d");
-
- proxmox_apt::config::init(APTConfig::new(
- Some(&test_dir.into_os_string().into_string().unwrap()),
- None,
- ));
+ let apt_lists_dir: PathBuf = test_dir.join("lists");
let absolute_suite_list = read_dir.join("absolute_suite.list");
let mut file = APTRepositoryFile::new(absolute_suite_list)?.unwrap();
file.parse()?;
- let infos = check_repositories(&[file], DebianCodename::Bullseye);
+ let infos = check_repositories(&[file], DebianCodename::Bullseye, &apt_lists_dir);
assert!(infos.is_empty());
let pve_list = read_dir.join("pve.list");
@@ -220,7 +214,7 @@ fn test_check_repositories() -> Result<(), Error> {
}
expected_infos.sort();
- let mut infos = check_repositories(&[file], DebianCodename::Bullseye);
+ let mut infos = check_repositories(&[file], DebianCodename::Bullseye, &apt_lists_dir);
infos.sort();
assert_eq!(infos, expected_infos);
@@ -286,7 +280,7 @@ fn test_check_repositories() -> Result<(), Error> {
}
expected_infos.sort();
- let mut infos = check_repositories(&[file], DebianCodename::Bullseye);
+ let mut infos = check_repositories(&[file], DebianCodename::Bullseye, &apt_lists_dir);
infos.sort();
assert_eq!(infos, expected_infos);
@@ -318,7 +312,7 @@ fn test_check_repositories() -> Result<(), Error> {
}
expected_infos.sort();
- let mut infos = check_repositories(&[file], DebianCodename::Bullseye);
+ let mut infos = check_repositories(&[file], DebianCodename::Bullseye, &apt_lists_dir);
infos.sort();
assert_eq!(infos, expected_infos);
@@ -329,11 +323,7 @@ fn test_check_repositories() -> Result<(), Error> {
fn test_get_cached_origin() -> Result<(), Error> {
let test_dir = std::env::current_dir()?.join("tests");
let read_dir = test_dir.join("sources.list.d");
-
- proxmox_apt::config::init(APTConfig::new(
- Some(&test_dir.into_os_string().into_string().unwrap()),
- None,
- ));
+ let apt_lists_dir: PathBuf = test_dir.clone().join("lists");
let pve_list = read_dir.join("pve.list");
let mut file = APTRepositoryFile::new(pve_list)?.unwrap();
@@ -351,7 +341,7 @@ fn test_get_cached_origin() -> Result<(), Error> {
assert_eq!(file.repositories.len(), origins.len());
for (n, repo) in file.repositories.iter().enumerate() {
- assert_eq!(repo.get_cached_origin()?, origins[n]);
+ assert_eq!(repo.get_cached_origin(&apt_lists_dir)?, origins[n]);
}
Ok(())
--
2.39.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config Wolfgang Bumiller
@ 2024-07-09 10:34 ` Fiona Ebner
2024-07-09 10:48 ` Wolfgang Bumiller
0 siblings, 1 reply; 8+ messages in thread
From: Fiona Ebner @ 2024-07-09 10:34 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Wolfgang Bumiller
Am 09.07.24 um 08:18 schrieb Wolfgang Bumiller:
> From: Dietmar Maurer <dietmar@proxmox.com>
>
> Because it was only used for the test setup. Instead, we simply
> add an apt_lists_dir parameter where we need it.
>
It was intended to be built upon and read in the actual values from the
user's apt configuration, i.e. the Dir::State and Dir::State::Lists
configuration variables. Just never got around to adding that feature.
Another option that was planned to be added is the
Dir::Ignore-Files-Silently one for
https://bugzilla.proxmox.com/show_bug.cgi?id=3510
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config
2024-07-09 10:34 ` Fiona Ebner
@ 2024-07-09 10:48 ` Wolfgang Bumiller
0 siblings, 0 replies; 8+ messages in thread
From: Wolfgang Bumiller @ 2024-07-09 10:48 UTC (permalink / raw)
To: Fiona Ebner; +Cc: Proxmox Backup Server development discussion
On Tue, Jul 09, 2024 at 12:34:37PM GMT, Fiona Ebner wrote:
> Am 09.07.24 um 08:18 schrieb Wolfgang Bumiller:
> > From: Dietmar Maurer <dietmar@proxmox.com>
> >
> > Because it was only used for the test setup. Instead, we simply
> > add an apt_lists_dir parameter where we need it.
> >
>
> It was intended to be built upon and read in the actual values from the
> user's apt configuration, i.e. the Dir::State and Dir::State::Lists
> configuration variables. Just never got around to adding that feature.
> Another option that was planned to be added is the
> Dir::Ignore-Files-Silently one for
> https://bugzilla.proxmox.com/show_bug.cgi?id=3510
If we want more flexibility in there it would probably make sense to
have some kind of top-level type which can be configured and takes care
of passing down the necessary data, with functionality being methods on
that - and get rid of all the freestanding functions and functions
returning longer tuples of not-self-describing data.
Would also make the generated rust docs a lot more natural to read.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-07-09 10:48 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-07-09 6:18 [pbs-devel] applied-series: [PATCH proxmox 1/6] apt-api-types: new crate Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 2/6] apt-api-types: use serde-plain to display/parse enums Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 3/6] apt: avoid direct impl on api types (use traits instead) Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 4/6] apt: use api types from apt-api-types crate Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 5/6] apt: add cache feature Wolfgang Bumiller
2024-07-09 6:18 ` [pbs-devel] [PATCH proxmox 6/6] apt: avoid global apt config Wolfgang Bumiller
2024-07-09 10:34 ` Fiona Ebner
2024-07-09 10:48 ` Wolfgang Bumiller
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.