From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox v8 1/9] s3 client: add crate for AWS s3 compatible object store client
Date: Tue, 15 Jul 2025 14:52:39 +0200 [thread overview]
Message-ID: <20250715125332.954494-2-c.ebner@proxmox.com> (raw)
In-Reply-To: <20250715125332.954494-1-c.ebner@proxmox.com>
Adds the client to connect to an AWS S3 compatible object store REST
API. Force the use of an TLS encrypted connection as the communication
with the object store will contain sensitive information.
For self-signed certificates, check the fingerprint against the one
configured. This follows along the lines of the PBS client, used to
connect to the PBS server API.
The `S3Client` stores the client state and has to be configured upon
instantiation by providing `S3ClientOptions`.
Adds the new config types `S3ClientConfig` and `S3ClientSecret` to
be used for configuration of datastore backends.
Secrets are stored as different config to never be returned on api
calls, only allowing to set/update the values.
Use a different name (`secrets_id`) for the unique identifier in case
of the secrets type, although the same id should be used for storing
and lookup. By this, clashing of property names when using flattened
types as api parameters is avoided.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
changes since version 7:
- Improved fingerprint check logic
- Add additional comments for cert chain check
- Add todo for shared cert handling logic once that has landed
Cargo.toml | 1 +
proxmox-s3-client/Cargo.toml | 32 ++++
proxmox-s3-client/debian/changelog | 5 +
proxmox-s3-client/debian/control | 81 +++++++++++
proxmox-s3-client/debian/copyright | 18 +++
proxmox-s3-client/debian/debcargo.toml | 7 +
proxmox-s3-client/src/api_types.rs | 172 ++++++++++++++++++++++
proxmox-s3-client/src/client.rs | 193 +++++++++++++++++++++++++
proxmox-s3-client/src/lib.rs | 12 ++
9 files changed, 521 insertions(+)
create mode 100644 proxmox-s3-client/Cargo.toml
create mode 100644 proxmox-s3-client/debian/changelog
create mode 100644 proxmox-s3-client/debian/control
create mode 100644 proxmox-s3-client/debian/copyright
create mode 100644 proxmox-s3-client/debian/debcargo.toml
create mode 100644 proxmox-s3-client/src/api_types.rs
create mode 100644 proxmox-s3-client/src/client.rs
create mode 100644 proxmox-s3-client/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index 020e7497..dc021796 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -33,6 +33,7 @@ members = [
"proxmox-router",
"proxmox-rrd",
"proxmox-rrd-api-types",
+ "proxmox-s3-client",
"proxmox-schema",
"proxmox-section-config",
"proxmox-sendmail",
diff --git a/proxmox-s3-client/Cargo.toml b/proxmox-s3-client/Cargo.toml
new file mode 100644
index 00000000..f62fc190
--- /dev/null
+++ b/proxmox-s3-client/Cargo.toml
@@ -0,0 +1,32 @@
+[package]
+name = "proxmox-s3-client"
+description = "low level REST API client for AWS S3 compatible object stores"
+version = "1.0.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+anyhow.workspace = true
+const_format.workspace = true
+hex = { workspace = true, features = [ "serde" ] }
+http-body-util.workspace = true
+hyper-util = { workspace = true, features = [ "client-legacy", "tokio", "http1" ] }
+hyper.workspace = true
+openssl.workspace = true
+regex.workspace = true
+serde.workspace = true
+tracing.workspace = true
+
+proxmox-base64.workspace = true
+proxmox-http = { workspace = true, features = [ "body", "client", "client-trait", "rate-limiter" ] }
+proxmox-schema = { workspace = true, features = [ "api-macro", "api-types" ] }
+
+[features]
+default = []
+impl = []
diff --git a/proxmox-s3-client/debian/changelog b/proxmox-s3-client/debian/changelog
new file mode 100644
index 00000000..b2696c33
--- /dev/null
+++ b/proxmox-s3-client/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-s3-client (1.0.0-1) bookworm; urgency=medium
+
+ * initial packaging
+
+ -- Proxmox Support Team <support@proxmox.com> Mon, 07 Jul 2025 09:33:10 +0200
diff --git a/proxmox-s3-client/debian/control b/proxmox-s3-client/debian/control
new file mode 100644
index 00000000..36b94143
--- /dev/null
+++ b/proxmox-s3-client/debian/control
@@ -0,0 +1,81 @@
+Source: rust-proxmox-s3-client
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.82) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-anyhow-1+default-dev <!nocheck>,
+ librust-const-format-0.2+default-dev <!nocheck>,
+ librust-hex-0.4+default-dev <!nocheck>,
+ librust-hex-0.4+serde-dev <!nocheck>,
+ librust-http-body-util-0.1+default-dev <!nocheck>,
+ librust-hyper-1+default-dev <!nocheck>,
+ librust-hyper-util-0.1+client-legacy-dev (>= 0.1.12-~~) <!nocheck>,
+ librust-hyper-util-0.1+default-dev (>= 0.1.12-~~) <!nocheck>,
+ librust-hyper-util-0.1+http1-dev (>= 0.1.12-~~) <!nocheck>,
+ librust-hyper-util-0.1+tokio-dev (>= 0.1.12-~~) <!nocheck>,
+ librust-openssl-0.10+default-dev <!nocheck>,
+ librust-proxmox-base64-1+default-dev <!nocheck>,
+ librust-proxmox-http-1+body-dev <!nocheck>,
+ librust-proxmox-http-1+client-dev <!nocheck>,
+ librust-proxmox-http-1+client-trait-dev <!nocheck>,
+ librust-proxmox-http-1+default-dev <!nocheck>,
+ librust-proxmox-http-1+rate-limiter-dev <!nocheck>,
+ librust-proxmox-schema-4+api-macro-dev (>= 4.1.0-~~) <!nocheck>,
+ librust-proxmox-schema-4+api-types-dev (>= 4.1.0-~~) <!nocheck>,
+ librust-proxmox-schema-4+default-dev (>= 4.1.0-~~) <!nocheck>,
+ librust-regex-1+default-dev (>= 1.5-~~) <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-tracing-0.1+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.0
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-s3-client
+Rules-Requires-Root: no
+
+Package: librust-proxmox-s3-client-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-anyhow-1+default-dev,
+ librust-const-format-0.2+default-dev,
+ librust-hex-0.4+default-dev,
+ librust-hex-0.4+serde-dev,
+ librust-http-body-util-0.1+default-dev,
+ librust-hyper-1+default-dev,
+ librust-hyper-util-0.1+client-legacy-dev (>= 0.1.12-~~),
+ librust-hyper-util-0.1+default-dev (>= 0.1.12-~~),
+ librust-hyper-util-0.1+http1-dev (>= 0.1.12-~~),
+ librust-hyper-util-0.1+tokio-dev (>= 0.1.12-~~),
+ librust-openssl-0.10+default-dev,
+ librust-proxmox-base64-1+default-dev,
+ librust-proxmox-http-1+body-dev,
+ librust-proxmox-http-1+client-dev,
+ librust-proxmox-http-1+client-trait-dev,
+ librust-proxmox-http-1+default-dev,
+ librust-proxmox-http-1+rate-limiter-dev,
+ librust-proxmox-schema-4+api-macro-dev (>= 4.1.0-~~),
+ librust-proxmox-schema-4+api-types-dev (>= 4.1.0-~~),
+ librust-proxmox-schema-4+default-dev (>= 4.1.0-~~),
+ librust-regex-1+default-dev (>= 1.5-~~),
+ librust-serde-1+default-dev,
+ librust-tracing-0.1+default-dev
+Provides:
+ librust-proxmox-s3-client+default-dev (= ${binary:Version}),
+ librust-proxmox-s3-client+impl-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1+default-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1+impl-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0+default-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0+impl-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0.0-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0.0+default-dev (= ${binary:Version}),
+ librust-proxmox-s3-client-1.0.0+impl-dev (= ${binary:Version})
+Description: Low level REST API client for AWS S3 compatible object stores - Rust source code
+ Source code for Debianized Rust crate "proxmox-s3-client"
diff --git a/proxmox-s3-client/debian/copyright b/proxmox-s3-client/debian/copyright
new file mode 100644
index 00000000..d6e3c304
--- /dev/null
+++ b/proxmox-s3-client/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2025 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/proxmox-s3-client/debian/debcargo.toml b/proxmox-s3-client/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-s3-client/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-s3-client/src/api_types.rs b/proxmox-s3-client/src/api_types.rs
new file mode 100644
index 00000000..ab0c1ec1
--- /dev/null
+++ b/proxmox-s3-client/src/api_types.rs
@@ -0,0 +1,172 @@
+use anyhow::bail;
+use const_format::concatcp;
+use serde::{Deserialize, Serialize};
+
+use proxmox_schema::api_types::{
+ CERT_FINGERPRINT_SHA256_SCHEMA, DNS_LABEL_STR, IPRE_STR, SAFE_ID_FORMAT,
+};
+use proxmox_schema::{api, const_regex, ApiStringFormat, Schema, StringSchema, Updater};
+
+#[rustfmt::skip]
+/// Regex to match S3 endpoint full qualified domain names, including template patterns for bucket
+/// name or region.
+pub const S3_ENDPOINT_NAME_STR: &str = concatcp!(
+ r"(?:(?:(", DNS_LABEL_STR, r"|\{\{bucket\}\}|\{\{region\}\})\.)*", DNS_LABEL_STR, ")"
+);
+
+const_regex! {
+ /// Regex to match S3 bucket names.
+ ///
+ /// Be as strict as possible following the rules as described here:
+ /// https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html#general-purpose-bucket-names
+ pub S3_BUCKET_NAME_REGEX = r"^[a-z0-9]([a-z0-9\-]*[a-z0-9])?$";
+ /// Regex to match S3 endpoints including template patterns.
+ pub S3_ENDPOINT_REGEX = concatcp!(r"^(?:", S3_ENDPOINT_NAME_STR, "|", IPRE_STR, r")$");
+ /// Regex to match S3 regions.
+ pub S3_REGION_REGEX = r"(^auto$)|(^[a-z]{2,}(?:-[a-z\d]+)+$)";
+}
+
+/// S3 REST API endpoint format.
+pub const S3_ENDPOINT_FORMAT: ApiStringFormat = ApiStringFormat::Pattern(&S3_ENDPOINT_REGEX);
+/// S3 region format.
+pub const S3_REGION_FORMAT: ApiStringFormat = ApiStringFormat::Pattern(&S3_REGION_REGEX);
+
+/// ID to uniquely identify an S3 client config.
+pub const S3_CLIENT_ID_SCHEMA: Schema =
+ StringSchema::new("ID to uniquely identify s3 client config.")
+ .format(&SAFE_ID_FORMAT)
+ .min_length(3)
+ .max_length(32)
+ .schema();
+
+/// Endpoint to access S3 object store.
+pub const S3_ENDPOINT_SCHEMA: Schema = StringSchema::new("Endpoint to access S3 object store.")
+ .format(&S3_ENDPOINT_FORMAT)
+ .schema();
+
+/// Region to access S3 object store.
+pub const S3_REGION_SCHEMA: Schema = StringSchema::new("Region to access S3 object store.")
+ .format(&S3_REGION_FORMAT)
+ .min_length(3)
+ .max_length(32)
+ .schema();
+
+/// Bucket to access S3 object store.
+pub const S3_BUCKET_NAME_SCHEMA: Schema = StringSchema::new("Bucket name for S3 object store.")
+ .format(&ApiStringFormat::VerifyFn(|bucket_name| {
+ if !(S3_BUCKET_NAME_REGEX.regex_obj)().is_match(bucket_name) {
+ bail!("Bucket name does not match the regex pattern");
+ }
+
+ // Exclude pre- and postfixes described here:
+ // https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html#general-purpose-bucket-names
+ let forbidden_prefixes = ["xn--", "sthree-", "amzn-s3-demo-"];
+ for prefix in forbidden_prefixes {
+ if bucket_name.starts_with(prefix) {
+ bail!("Bucket name cannot start with '{prefix}'");
+ }
+ }
+
+ let forbidden_postfixes = ["--ol-s3", ".mrap", "--x-s3"];
+ for postfix in forbidden_postfixes {
+ if bucket_name.ends_with(postfix) {
+ bail!("Bucket name cannot end with '{postfix}'");
+ }
+ }
+
+ Ok(())
+ }))
+ .min_length(3)
+ .max_length(63)
+ .schema();
+
+#[api(
+ properties: {
+ id: {
+ schema: S3_CLIENT_ID_SCHEMA,
+ },
+ endpoint: {
+ schema: S3_ENDPOINT_SCHEMA,
+ },
+ port: {
+ type: u16,
+ optional: true,
+ },
+ region: {
+ schema: S3_REGION_SCHEMA,
+ optional: true,
+ },
+ fingerprint: {
+ schema: CERT_FINGERPRINT_SHA256_SCHEMA,
+ optional: true,
+ },
+ "access-key": {
+ type: String,
+ },
+ "path-style": {
+ type: bool,
+ optional: true,
+ default: false,
+ },
+ "put-rate-limit": {
+ type: u64,
+ optional: true,
+ },
+ }
+)]
+#[derive(Serialize, Deserialize, Updater, Clone, PartialEq)]
+#[serde(rename_all = "kebab-case")]
+/// S3 client configuration properties.
+pub struct S3ClientConfig {
+ /// ID to identify s3 client config.
+ #[updater(skip)]
+ pub id: String,
+ /// Endpoint to access S3 object store.
+ pub endpoint: String,
+ /// Port to access S3 object store.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub port: Option<u16>,
+ /// Region to access S3 object store.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub region: Option<String>,
+ /// Access key for S3 object store.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub fingerprint: Option<String>,
+ /// Access key for S3 object store.
+ pub access_key: String,
+ /// Use path style bucket addressing over vhost style.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub path_style: Option<bool>,
+ /// Rate limit for put requests given as #reqest/s.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub put_rate_limit: Option<u64>,
+}
+
+impl S3ClientConfig {
+ /// Helper method to get ACL path for S3 client config
+ pub fn acl_path(&self) -> Vec<&str> {
+ // Needs permissions on root path
+ Vec::new()
+ }
+}
+
+#[api(
+ properties: {
+ "secrets-id": {
+ type: String,
+ },
+ "secret-key": {
+ type: String,
+ },
+ }
+)]
+#[derive(Serialize, Deserialize, Updater, Clone, PartialEq)]
+#[serde(rename_all = "kebab-case")]
+/// S3 client secrets configuration properties.
+pub struct S3ClientSecretsConfig {
+ /// ID to identify s3 client secret config.
+ #[updater(skip)]
+ pub secrets_id: String,
+ /// Secret key for S3 object store.
+ pub secret_key: String,
+}
diff --git a/proxmox-s3-client/src/client.rs b/proxmox-s3-client/src/client.rs
new file mode 100644
index 00000000..1ba42a08
--- /dev/null
+++ b/proxmox-s3-client/src/client.rs
@@ -0,0 +1,193 @@
+use std::sync::{Arc, Mutex};
+use std::time::Duration;
+
+use anyhow::{bail, format_err, Context, Error};
+use hyper::http::uri::Authority;
+use hyper_util::client::legacy::connect::HttpConnector;
+use hyper_util::client::legacy::Client;
+use hyper_util::rt::TokioExecutor;
+use openssl::hash::MessageDigest;
+use openssl::ssl::{SslConnector, SslMethod, SslVerifyMode};
+use openssl::x509::X509StoreContextRef;
+use tracing::error;
+
+use proxmox_http::client::HttpsConnector;
+use proxmox_http::{Body, RateLimiter};
+use proxmox_schema::api_types::CERT_FINGERPRINT_SHA256_SCHEMA;
+
+use crate::api_types::{S3ClientConfig, S3ClientSecretsConfig};
+
+const S3_HTTP_CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
+const S3_TCP_KEEPALIVE_TIME: u32 = 120;
+
+/// Configuration options for client
+pub struct S3ClientOptions {
+ /// Endpoint to access S3 object store.
+ pub endpoint: String,
+ /// Port to access S3 object store.
+ pub port: Option<u16>,
+ /// Bucket to access S3 object store.
+ pub bucket: String,
+ /// Common prefix within bucket to use for objects keys for this client instance.
+ pub common_prefix: String,
+ /// Use path style bucket addressing over vhost style.
+ pub path_style: bool,
+ /// Secret key for S3 object store.
+ pub secret_key: String,
+ /// Access key for S3 object store.
+ pub access_key: String,
+ /// Region to access S3 object store.
+ pub region: String,
+ /// API certificate fingerprint for self signed certificates.
+ pub fingerprint: Option<String>,
+ /// Rate limit for put requests given as #reqest/s.
+ pub put_rate_limit: Option<u64>,
+}
+
+impl S3ClientOptions {
+ /// Construct options for the S3 client give the provided configuration parameters.
+ pub fn from_config(
+ config: S3ClientConfig,
+ secrets: S3ClientSecretsConfig,
+ bucket: String,
+ common_prefix: String,
+ ) -> Self {
+ Self {
+ endpoint: config.endpoint,
+ port: config.port,
+ bucket,
+ common_prefix,
+ path_style: config.path_style.unwrap_or_default(),
+ region: config.region.unwrap_or("us-west-1".to_string()),
+ fingerprint: config.fingerprint,
+ access_key: config.access_key,
+ secret_key: secrets.secret_key,
+ put_rate_limit: config.put_rate_limit,
+ }
+ }
+}
+
+/// S3 client for object stores compatible with the AWS S3 API
+pub struct S3Client {
+ client: Client<HttpsConnector, Body>,
+ options: S3ClientOptions,
+ authority: Authority,
+ put_rate_limiter: Option<Arc<Mutex<RateLimiter>>>,
+}
+
+impl S3Client {
+ /// Creates a new S3 client instance, connecting to the provided endpoint using https given the
+ /// provided options.
+ pub fn new(options: S3ClientOptions) -> Result<Self, Error> {
+ let expected_fingerprint = if let Some(ref fingerprint) = options.fingerprint {
+ CERT_FINGERPRINT_SHA256_SCHEMA
+ .unwrap_string_schema()
+ .check_constraints(fingerprint)
+ .context("invalid fingerprint provided")?;
+ Some(fingerprint.to_lowercase())
+ } else {
+ None
+ };
+ let verified_fingerprint = Arc::new(Mutex::new(None));
+ let trust_openssl_valid = Arc::new(Mutex::new(true));
+ let mut ssl_connector_builder = SslConnector::builder(SslMethod::tls())?;
+ ssl_connector_builder.set_verify_callback(
+ SslVerifyMode::PEER,
+ move |openssl_valid, context| match Self::verify_certificate_fingerprint(
+ openssl_valid,
+ context,
+ expected_fingerprint.clone(),
+ trust_openssl_valid.clone(),
+ ) {
+ Ok(None) => true,
+ Ok(Some(fingerprint)) => {
+ *verified_fingerprint.lock().unwrap() = Some(fingerprint);
+ true
+ }
+ Err(err) => {
+ error!("certificate validation failed {err:#}");
+ false
+ }
+ },
+ );
+
+ let mut http_connector = HttpConnector::new();
+ // want communication to object store backend api to always use https
+ http_connector.enforce_http(false);
+ http_connector.set_connect_timeout(Some(S3_HTTP_CONNECT_TIMEOUT));
+ let https_connector = HttpsConnector::with_connector(
+ http_connector,
+ ssl_connector_builder.build(),
+ S3_TCP_KEEPALIVE_TIME,
+ );
+ let client = Client::builder(TokioExecutor::new()).build::<_, Body>(https_connector);
+
+ let authority_template = if let Some(port) = options.port {
+ format!("{}:{port}", options.endpoint)
+ } else {
+ options.endpoint.clone()
+ };
+ let authority = authority_template
+ .replace("{{bucket}}", &options.bucket)
+ .replace("{{region}}", &options.region);
+ let authority = Authority::try_from(authority)?;
+
+ let put_rate_limiter = options.put_rate_limit.map(|limit| {
+ let limiter = RateLimiter::new(limit, limit);
+ Arc::new(Mutex::new(limiter))
+ });
+
+ Ok(Self {
+ client,
+ options,
+ authority,
+ put_rate_limiter,
+ })
+ }
+
+ // TODO: replace with our shared TLS cert verification once available
+ fn verify_certificate_fingerprint(
+ openssl_valid: bool,
+ context: &mut X509StoreContextRef,
+ expected_fingerprint: Option<String>,
+ trust_openssl: Arc<Mutex<bool>>,
+ ) -> Result<Option<String>, Error> {
+ let mut trust_openssl_valid = trust_openssl.lock().unwrap();
+
+ // only rely on openssl prevalidation if was not forced earlier
+ if openssl_valid && *trust_openssl_valid {
+ return Ok(None);
+ }
+
+ let certificate = match context.current_cert() {
+ Some(certificate) => certificate,
+ None => bail!("context lacks current certificate."),
+ };
+
+ // force trust in case of a chain, but set flag to no longer trust prevalidation by openssl
+ // see https://bugzilla.proxmox.com/show_bug.cgi?id=5248
+ if context.error_depth() > 0 {
+ *trust_openssl_valid = false;
+ return Ok(None);
+ }
+
+ let certificate_digest = certificate
+ .digest(MessageDigest::sha256())
+ .context("failed to calculate certificate digest")?;
+ let certificate_fingerprint = certificate_digest
+ .iter()
+ .map(|byte| format!("{byte:02x}"))
+ .collect::<Vec<String>>()
+ .join(":");
+
+ if let Some(expected_fingerprint) = expected_fingerprint {
+ if expected_fingerprint == certificate_fingerprint {
+ return Ok(Some(certificate_fingerprint));
+ }
+ }
+
+ Err(format_err!(
+ "unexpected certificate fingerprint {certificate_fingerprint}"
+ ))
+ }
+}
diff --git a/proxmox-s3-client/src/lib.rs b/proxmox-s3-client/src/lib.rs
new file mode 100644
index 00000000..e579ffbb
--- /dev/null
+++ b/proxmox-s3-client/src/lib.rs
@@ -0,0 +1,12 @@
+//! Low level REST API client for AWS S3 compatible object stores
+#![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))]
+#![deny(unsafe_op_in_unsafe_fn)]
+#![deny(missing_docs)]
+
+mod api_types;
+pub use api_types::*;
+
+#[cfg(feature = "impl")]
+mod client;
+#[cfg(feature = "impl")]
+pub use client::{S3Client, S3ClientOptions};
--
2.47.2
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
next prev parent reply other threads:[~2025-07-15 12:54 UTC|newest]
Thread overview: 108+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-15 12:52 [pbs-devel] [PATCH proxmox{, -backup} v8 00/54] fix #2943: S3 storage backend for datastores Christian Ebner
2025-07-15 12:52 ` Christian Ebner [this message]
2025-07-15 21:13 ` [pbs-devel] partially-applied-series: [PATCH proxmox v8 1/9] s3 client: add crate for AWS s3 compatible object store client Thomas Lamprecht
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 2/9] s3 client: implement AWS signature v4 request authentication Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 3/9] s3 client: add dedicated type for s3 object keys Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 4/9] s3 client: add type for last modified timestamp in responses Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 5/9] s3 client: add helper to parse http date headers Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 6/9] s3 client: implement methods to operate on s3 objects in bucket Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 7/9] s3 client: add example usage for basic operations Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 8/9] pbs-api-types: extend datastore config by backend config enum Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox v8 9/9] pbs-api-types: maintenance: add new maintenance mode S3 refresh Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 01/45] datastore: add helpers for path/digest to s3 object key conversion Christian Ebner
2025-07-18 7:24 ` Lukas Wagner
2025-07-18 8:34 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 02/45] config: introduce s3 object store client configuration Christian Ebner
2025-07-18 7:22 ` Lukas Wagner
2025-07-18 8:37 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 03/45] api: config: implement endpoints to manipulate and list s3 configs Christian Ebner
2025-07-18 7:32 ` Lukas Wagner
2025-07-18 8:40 ` Christian Ebner
2025-07-18 9:07 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 04/45] api: datastore: check s3 backend bucket access on datastore create Christian Ebner
2025-07-18 7:40 ` Lukas Wagner
2025-07-18 8:55 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 05/45] api/cli: add endpoint and command to check s3 client connection Christian Ebner
2025-07-18 7:43 ` Lukas Wagner
2025-07-18 9:04 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 06/45] datastore: allow to get the backend for a datastore Christian Ebner
2025-07-18 7:52 ` Lukas Wagner
2025-07-18 9:10 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 07/45] api: backup: store datastore backend in runtime environment Christian Ebner
2025-07-18 7:54 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 08/45] api: backup: conditionally upload chunks to s3 object store backend Christian Ebner
2025-07-18 8:11 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 09/45] api: backup: conditionally upload blobs " Christian Ebner
2025-07-18 8:13 ` Lukas Wagner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 10/45] api: backup: conditionally upload indices " Christian Ebner
2025-07-18 8:20 ` Lukas Wagner
2025-07-18 9:24 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 11/45] api: backup: conditionally upload manifest " Christian Ebner
2025-07-18 8:26 ` Lukas Wagner
2025-07-18 9:33 ` Christian Ebner
2025-07-15 12:52 ` [pbs-devel] [PATCH proxmox-backup v8 12/45] api: datastore: conditionally upload client log to s3 backend Christian Ebner
2025-07-18 8:28 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 13/45] sync: pull: conditionally upload content " Christian Ebner
2025-07-18 8:35 ` Lukas Wagner
2025-07-18 9:43 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 14/45] api: reader: fetch chunks based on datastore backend Christian Ebner
2025-07-18 8:38 ` Lukas Wagner
2025-07-18 9:58 ` Christian Ebner
2025-07-18 10:03 ` Lukas Wagner
2025-07-18 10:10 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 15/45] datastore: local chunk reader: read chunks based on backend Christian Ebner
2025-07-18 8:45 ` Lukas Wagner
2025-07-18 10:11 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 16/45] verify worker: add datastore backed to verify worker Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 17/45] verify: implement chunk verification for stores with s3 backend Christian Ebner
2025-07-18 8:56 ` Lukas Wagner
2025-07-18 11:45 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 18/45] datastore: create namespace marker in " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 19/45] datastore: create/delete protected marker file on s3 storage backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 20/45] datastore: prune groups/snapshots from s3 object store backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 21/45] datastore: get and set owner for s3 " Christian Ebner
2025-07-18 9:25 ` Lukas Wagner
2025-07-18 12:12 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 22/45] datastore: implement garbage collection for s3 backend Christian Ebner
2025-07-18 9:47 ` Lukas Wagner
2025-07-18 14:31 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 23/45] ui: add datastore type selector and reorganize component layout Christian Ebner
2025-07-18 9:55 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 24/45] ui: add s3 client edit window for configuration create/edit Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 25/45] ui: add s3 client view for configuration Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 26/45] ui: expose the s3 client view in the navigation tree Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 27/45] ui: add s3 client selector and bucket field for s3 backend setup Christian Ebner
2025-07-18 10:02 ` Lukas Wagner
2025-07-19 12:28 ` Christian Ebner
2025-07-22 9:25 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 28/45] tools: lru cache: add removed callback for evicted cache nodes Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 29/45] tools: async lru cache: implement insert, remove and contains methods Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 30/45] datastore: add local datastore cache for network attached storages Christian Ebner
2025-07-18 11:24 ` Lukas Wagner
2025-07-18 14:59 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 31/45] api: backup: use local datastore cache on s3 backend chunk upload Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 32/45] api: reader: use local datastore cache on s3 backend chunk fetching Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 33/45] datastore: local chunk reader: get cached chunk from local cache store Christian Ebner
2025-07-18 11:36 ` Lukas Wagner
2025-07-18 15:04 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 34/45] api: backup: add no-cache flag to bypass local datastore cache Christian Ebner
2025-07-18 11:41 ` Lukas Wagner
2025-07-18 15:37 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 35/45] api/datastore: implement refresh endpoint for stores with s3 backend Christian Ebner
2025-07-18 12:01 ` Lukas Wagner
2025-07-18 15:51 ` Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 36/45] cli: add dedicated subcommand for datastore s3 refresh Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 37/45] ui: render s3 refresh as valid maintenance type and task description Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 38/45] ui: expose s3 refresh button for datastores backed by object store Christian Ebner
2025-07-18 12:46 ` Lukas Wagner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 39/45] datastore: conditionally upload atime marker chunk to s3 backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 40/45] bin: implement client subcommands for s3 configuration manipulation Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 41/45] bin: expose reuse-datastore flag for proxmox-backup-manager Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 42/45] datastore: mark store as in-use by setting marker on s3 backend Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 43/45] datastore: run s3-refresh when reusing a datastore with " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 44/45] api/ui: add flag to allow overwriting in-use marker for " Christian Ebner
2025-07-15 12:53 ` [pbs-devel] [PATCH proxmox-backup v8 45/45] docs: Add section describing how to setup s3 backed datastore Christian Ebner
2025-07-18 13:14 ` Maximiliano Sandoval
2025-07-18 14:38 ` Christian Ebner
2025-07-18 13:16 ` [pbs-devel] [PATCH proxmox{, -backup} v8 00/54] fix #2943: S3 storage backend for datastores Lukas Wagner
2025-07-19 12:52 ` [pbs-devel] superseded: " Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250715125332.954494-2-c.ebner@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox