* [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate
@ 2021-09-21 5:58 Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 01/16] start new proxmox-rest-server workspace Dietmar Maurer
` (16 more replies)
0 siblings, 17 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
Changes in v2:
- renamed the crate from "pbs-server" into "proxmox-rest-server"
- depend on proxmox "0.13.3"
- worker_state: move tasktype() code to src/api2/node/tasks.rs
- dependency fixes for proxmox_restore_daemon crate (fabian)
Dietmar Maurer (16):
start new proxmox-rest-server workspace
move ApiConfig, FileLogger and CommandoSocket to proxmox-rest-server
workspace
move src/tools/daemon.rs to proxmox-rest-server workspace
move src/server/environment.rs to proxmox-rest-server crate
move src/server/formatter.rs to proxmox-rest-server crate
move src/tools/compression.rs to proxmox-rest-server crate
move normalize_uri_path and extract_cookie to proxmox-rest-server
crate
rest server: simplify get_index() method signature
make get_index and ApiConfig property (callback)
rest server: return UserInformation from ApiAuth::check_auth
rest server: do not use pbs_api_types::Authid
rest server: cleanup auth-log handling
move src/server/rest.rs to proxmox-rest-server crate
move proxmox_restore_daemon code into extra crate
basically a (semantic) revert of commit
991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround
linkage issues from openid/curl build server stuff separate"
worker_state: move tasktype() code to src/api2/node/tasks.rs
Cargo.toml | 5 +-
Makefile | 10 +-
pbs-api-types/Cargo.toml | 2 +-
pbs-client/Cargo.toml | 2 +-
pbs-config/Cargo.toml | 2 +-
pbs-datastore/Cargo.toml | 2 +-
pbs-fuse-loop/Cargo.toml | 2 +-
pbs-systemd/Cargo.toml | 2 +-
pbs-tape/Cargo.toml | 2 +-
pbs-tools/Cargo.toml | 2 +-
proxmox-backup-client/Cargo.toml | 2 +-
proxmox-backup-debug/Cargo.toml | 2 +-
proxmox-file-restore/Cargo.toml | 2 +-
proxmox-rest-server/Cargo.toml | 30 ++++
.../src/api_config.rs | 76 +++++++--
.../src}/command_socket.rs | 18 ++-
.../src}/compression.rs | 0
.../src}/daemon.rs | 14 +-
proxmox-rest-server/src/environment.rs | 94 +++++++++++
.../src}/file_logger.rs | 46 +++---
.../src}/formatter.rs | 0
proxmox-rest-server/src/lib.rs | 141 ++++++++++++++++
.../src}/rest.rs | 147 +++++------------
.../src}/state.rs | 4 +-
proxmox-restore-daemon/Cargo.toml | 36 +++++
.../src/main.rs | 24 ++-
.../src}/proxmox_restore_daemon/api.rs | 0
.../src}/proxmox_restore_daemon/auth.rs | 20 ++-
.../src}/proxmox_restore_daemon/disk.rs | 0
.../src}/proxmox_restore_daemon/mod.rs | 0
.../src}/proxmox_restore_daemon/watchdog.rs | 0
pxar-bin/Cargo.toml | 2 +-
src/api2/access/mod.rs | 38 ++---
src/api2/access/openid.rs | 134 ++++++++-------
src/api2/admin/datastore.rs | 5 +-
src/api2/backup/environment.rs | 2 +-
src/api2/node/mod.rs | 6 +-
src/api2/node/tasks.rs | 11 +-
src/api2/reader/environment.rs | 2 +-
src/backup/datastore.rs | 3 +-
src/backup/verify.rs | 6 +-
src/bin/proxmox-backup-api.rs | 56 ++++++-
src/bin/proxmox-backup-proxy.rs | 153 +++++++++++++++---
src/server/auth.rs | 39 ++---
src/server/environment.rs | 53 ------
src/server/h2service.rs | 9 +-
src/server/mod.rs | 21 +--
src/server/worker_task.rs | 33 ++--
src/tools/mod.rs | 98 -----------
tests/worker-task-abort.rs | 9 +-
50 files changed, 833 insertions(+), 534 deletions(-)
create mode 100644 proxmox-rest-server/Cargo.toml
rename src/server/config.rs => proxmox-rest-server/src/api_config.rs (68%)
rename {src/server => proxmox-rest-server/src}/command_socket.rs (94%)
rename {src/tools => proxmox-rest-server/src}/compression.rs (100%)
rename {src/tools => proxmox-rest-server/src}/daemon.rs (97%)
create mode 100644 proxmox-rest-server/src/environment.rs
rename {src/tools => proxmox-rest-server/src}/file_logger.rs (81%)
rename {src/server => proxmox-rest-server/src}/formatter.rs (100%)
create mode 100644 proxmox-rest-server/src/lib.rs
rename {src/server => proxmox-rest-server/src}/rest.rs (85%)
rename {src/server => proxmox-rest-server/src}/state.rs (97%)
create mode 100644 proxmox-restore-daemon/Cargo.toml
rename src/bin/proxmox-restore-daemon.rs => proxmox-restore-daemon/src/main.rs (90%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/api.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/auth.rs (63%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/disk.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/mod.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/watchdog.rs (100%)
delete mode 100644 src/server/environment.rs
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 01/16] start new proxmox-rest-server workspace
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 02/16] move ApiConfig, FileLogger and CommandoSocket to " Dietmar Maurer
` (15 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
Cargo.toml | 2 ++
Makefile | 1 +
proxmox-rest-server/Cargo.toml | 9 +++++++++
proxmox-rest-server/src/lib.rs | 0
4 files changed, 12 insertions(+)
create mode 100644 proxmox-rest-server/Cargo.toml
create mode 100644 proxmox-rest-server/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index 72d786c9..913dc98e 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -26,6 +26,7 @@ members = [
"pbs-datastore",
"pbs-fuse-loop",
"pbs-runtime",
+ "proxmox-rest-server",
"pbs-systemd",
"pbs-tape",
"pbs-tools",
@@ -107,6 +108,7 @@ pbs-client = { path = "pbs-client" }
pbs-config = { path = "pbs-config" }
pbs-datastore = { path = "pbs-datastore" }
pbs-runtime = { path = "pbs-runtime" }
+proxmox-rest-server = { path = "proxmox-rest-server" }
pbs-systemd = { path = "pbs-systemd" }
pbs-tools = { path = "pbs-tools" }
pbs-tape = { path = "pbs-tape" }
diff --git a/Makefile b/Makefile
index e6c85a2e..b1d1ea93 100644
--- a/Makefile
+++ b/Makefile
@@ -39,6 +39,7 @@ SUBCRATES := \
pbs-datastore \
pbs-fuse-loop \
pbs-runtime \
+ proxmox-rest-server \
pbs-systemd \
pbs-tape \
pbs-tools \
diff --git a/proxmox-rest-server/Cargo.toml b/proxmox-rest-server/Cargo.toml
new file mode 100644
index 00000000..836d9e28
--- /dev/null
+++ b/proxmox-rest-server/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "proxmox-rest-server"
+version = "0.1.0"
+authors = ["Proxmox Support Team <support@proxmox.com>"]
+edition = "2018"
+description = "REST server implementation"
+
+[dependencies]
+anyhow = "1.0"
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
new file mode 100644
index 00000000..e69de29b
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 02/16] move ApiConfig, FileLogger and CommandoSocket to proxmox-rest-server workspace
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 01/16] start new proxmox-rest-server workspace Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 03/16] move src/tools/daemon.rs " Dietmar Maurer
` (14 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
ApiConfig: avoid using pbs_config::backup_user()
CommandoSocket: avoid using pbs_config::backup_user()
FileLogger: avoid using pbs_config::backup_user()
- use atomic_open_or_create_file()
Auth Trait: moved definitions to proxmox-rest-server/src/lib.rs
- removed CachedUserInfo patrameter
- return user as String (not Authid)
---
Cargo.toml | 2 +-
pbs-api-types/Cargo.toml | 2 +-
pbs-client/Cargo.toml | 2 +-
pbs-config/Cargo.toml | 2 +-
pbs-datastore/Cargo.toml | 2 +-
pbs-fuse-loop/Cargo.toml | 2 +-
pbs-systemd/Cargo.toml | 2 +-
pbs-tape/Cargo.toml | 2 +-
pbs-tools/Cargo.toml | 2 +-
proxmox-backup-client/Cargo.toml | 2 +-
proxmox-backup-debug/Cargo.toml | 2 +-
proxmox-file-restore/Cargo.toml | 2 +-
proxmox-rest-server/Cargo.toml | 15 ++++++
.../src/api_config.rs | 13 +++--
.../src}/command_socket.rs | 18 ++++---
.../src}/file_logger.rs | 46 +++++++++-------
proxmox-rest-server/src/lib.rs | 54 +++++++++++++++++++
.../src}/state.rs | 4 +-
pxar-bin/Cargo.toml | 2 +-
src/api2/admin/datastore.rs | 2 +-
src/api2/node/mod.rs | 4 +-
src/backup/datastore.rs | 3 +-
src/backup/verify.rs | 6 +--
src/bin/proxmox-backup-api.rs | 23 +++++---
src/bin/proxmox-backup-proxy.rs | 37 ++++++++-----
src/bin/proxmox-restore-daemon.rs | 4 +-
src/bin/proxmox_restore_daemon/auth.rs | 10 ++--
src/server/auth.rs | 33 +++---------
src/server/mod.rs | 13 +----
src/server/rest.rs | 25 +++++----
src/server/worker_task.rs | 22 ++++----
src/tools/daemon.rs | 9 ++--
src/tools/mod.rs | 24 ---------
tests/worker-task-abort.rs | 9 ++--
34 files changed, 223 insertions(+), 177 deletions(-)
rename src/server/config.rs => proxmox-rest-server/src/api_config.rs (92%)
rename {src/server => proxmox-rest-server/src}/command_socket.rs (94%)
rename {src/tools => proxmox-rest-server/src}/file_logger.rs (81%)
rename {src/server => proxmox-rest-server/src}/state.rs (97%)
diff --git a/Cargo.toml b/Cargo.toml
index 913dc98e..42528a49 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -96,7 +96,7 @@ zstd = { version = "0.6", features = [ "bindgen" ] }
pathpatterns = "0.1.2"
pxar = { version = "0.10.1", features = [ "tokio-io" ] }
-proxmox = { version = "0.13.0", features = [ "sortable-macro", "api-macro", "cli", "router", "tfa" ] }
+proxmox = { version = "0.13.3", features = [ "sortable-macro", "api-macro", "cli", "router", "tfa" ] }
proxmox-acme-rs = "0.2.1"
proxmox-apt = "0.7.0"
proxmox-http = { version = "0.4.0", features = [ "client", "http-helpers", "websocket" ] }
diff --git a/pbs-api-types/Cargo.toml b/pbs-api-types/Cargo.toml
index 15507328..02c8c2d4 100644
--- a/pbs-api-types/Cargo.toml
+++ b/pbs-api-types/Cargo.toml
@@ -14,7 +14,7 @@ openssl = "0.10"
regex = "1.2"
serde = { version = "1.0", features = ["derive"] }
-proxmox = { version = "0.13.0", default-features = false, features = [ "api-macro" ] }
+proxmox = { version = "0.13.3", default-features = false, features = [ "api-macro" ] }
pbs-systemd = { path = "../pbs-systemd" }
pbs-tools = { path = "../pbs-tools" }
diff --git a/pbs-client/Cargo.toml b/pbs-client/Cargo.toml
index fb12636f..076acfb5 100644
--- a/pbs-client/Cargo.toml
+++ b/pbs-client/Cargo.toml
@@ -28,7 +28,7 @@ tower-service = "0.3.0"
xdg = "2.2"
pathpatterns = "0.1.2"
-proxmox = { version = "0.13.0", default-features = false, features = [ "cli" ] }
+proxmox = { version = "0.13.3", default-features = false, features = [ "cli" ] }
proxmox-fuse = "0.1.1"
proxmox-http = { version = "0.4.0", features = [ "client", "http-helpers", "websocket" ] }
pxar = { version = "0.10.1", features = [ "tokio-io" ] }
diff --git a/pbs-config/Cargo.toml b/pbs-config/Cargo.toml
index 7f4258bd..48830045 100644
--- a/pbs-config/Cargo.toml
+++ b/pbs-config/Cargo.toml
@@ -16,7 +16,7 @@ nix = "0.19.1"
regex = "1.2"
once_cell = "1.3.1"
-proxmox = { version = "0.13.0", default-features = false, features = [ "cli" ] }
+proxmox = { version = "0.13.3", default-features = false, features = [ "cli" ] }
pbs-api-types = { path = "../pbs-api-types" }
pbs-buildcfg = { path = "../pbs-buildcfg" }
diff --git a/pbs-datastore/Cargo.toml b/pbs-datastore/Cargo.toml
index 32eae0d7..5b3c7fab 100644
--- a/pbs-datastore/Cargo.toml
+++ b/pbs-datastore/Cargo.toml
@@ -23,7 +23,7 @@ zstd = { version = "0.6", features = [ "bindgen" ] }
pathpatterns = "0.1.2"
pxar = "0.10.1"
-proxmox = { version = "0.13.0", default-features = false, features = [ "api-macro" ] }
+proxmox = { version = "0.13.3", default-features = false, features = [ "api-macro" ] }
pbs-api-types = { path = "../pbs-api-types" }
pbs-tools = { path = "../pbs-tools" }
diff --git a/pbs-fuse-loop/Cargo.toml b/pbs-fuse-loop/Cargo.toml
index 5865a463..c3220be7 100644
--- a/pbs-fuse-loop/Cargo.toml
+++ b/pbs-fuse-loop/Cargo.toml
@@ -14,7 +14,7 @@ nix = "0.19.1"
regex = "1.2"
tokio = { version = "1.6", features = [] }
-proxmox = "0.13.0"
+proxmox = "0.13.3"
proxmox-fuse = "0.1.1"
pbs-tools = { path = "../pbs-tools" }
diff --git a/pbs-systemd/Cargo.toml b/pbs-systemd/Cargo.toml
index fcb60445..b4575f0a 100644
--- a/pbs-systemd/Cargo.toml
+++ b/pbs-systemd/Cargo.toml
@@ -11,6 +11,6 @@ bitflags = "1.2.1"
lazy_static = "1.4"
nom = "5.1"
-proxmox = { version = "0.13.0", default-features = false }
+proxmox = { version = "0.13.3", default-features = false }
pbs-tools = { path = "../pbs-tools" }
diff --git a/pbs-tape/Cargo.toml b/pbs-tape/Cargo.toml
index 719ef01c..4ffae21e 100644
--- a/pbs-tape/Cargo.toml
+++ b/pbs-tape/Cargo.toml
@@ -18,7 +18,7 @@ bitflags = "1.2.1"
regex = "1.2"
udev = ">= 0.3, <0.5"
-proxmox = { version = "0.13.0", default-features = false, features = [] }
+proxmox = { version = "0.13.3", default-features = false, features = [] }
pbs-api-types = { path = "../pbs-api-types" }
pbs-tools = { path = "../pbs-tools" }
diff --git a/pbs-tools/Cargo.toml b/pbs-tools/Cargo.toml
index 89c6303c..88f6f54c 100644
--- a/pbs-tools/Cargo.toml
+++ b/pbs-tools/Cargo.toml
@@ -30,7 +30,7 @@ url = "2.1"
walkdir = "2"
zstd = { version = "0.6", features = [ "bindgen" ] }
-proxmox = { version = "0.13.0", default-features = false, features = [ "tokio" ] }
+proxmox = { version = "0.13.3", default-features = false, features = [ "tokio" ] }
pbs-buildcfg = { path = "../pbs-buildcfg" }
pbs-runtime = { path = "../pbs-runtime" }
diff --git a/proxmox-backup-client/Cargo.toml b/proxmox-backup-client/Cargo.toml
index b1ecf3e4..6c1bb936 100644
--- a/proxmox-backup-client/Cargo.toml
+++ b/proxmox-backup-client/Cargo.toml
@@ -22,7 +22,7 @@ zstd = { version = "0.6", features = [ "bindgen" ] }
pathpatterns = "0.1.2"
pxar = { version = "0.10.1", features = [ "tokio-io" ] }
-proxmox = { version = "0.13.0", features = [ "sortable-macro", "api-macro", "cli", "router" ] }
+proxmox = { version = "0.13.3", features = [ "sortable-macro", "api-macro", "cli", "router" ] }
pbs-api-types = { path = "../pbs-api-types" }
pbs-buildcfg = { path = "../pbs-buildcfg" }
diff --git a/proxmox-backup-debug/Cargo.toml b/proxmox-backup-debug/Cargo.toml
index 7f1f596d..21b3cc2f 100644
--- a/proxmox-backup-debug/Cargo.toml
+++ b/proxmox-backup-debug/Cargo.toml
@@ -9,7 +9,7 @@ anyhow = "1.0"
walkdir = "2"
serde_json = "1.0"
-proxmox = { version = "0.13.0", features = [ "api-macro", "cli" ] }
+proxmox = { version = "0.13.3", features = [ "api-macro", "cli" ] }
pbs-config = { path = "../pbs-config" }
pbs-client = { path = "../pbs-client" }
diff --git a/proxmox-file-restore/Cargo.toml b/proxmox-file-restore/Cargo.toml
index 127397b6..97f9c414 100644
--- a/proxmox-file-restore/Cargo.toml
+++ b/proxmox-file-restore/Cargo.toml
@@ -16,7 +16,7 @@ tokio = { version = "1.6", features = [ "io-std", "rt", "rt-multi-thread", "time
pxar = { version = "0.10.1", features = [ "tokio-io" ] }
-proxmox = { version = "0.13.0", features = [ "api-macro", "cli" ] }
+proxmox = { version = "0.13.3", features = [ "api-macro", "cli" ] }
pbs-api-types = { path = "../pbs-api-types" }
pbs-buildcfg = { path = "../pbs-buildcfg" }
diff --git a/proxmox-rest-server/Cargo.toml b/proxmox-rest-server/Cargo.toml
index 836d9e28..33ed6f39 100644
--- a/proxmox-rest-server/Cargo.toml
+++ b/proxmox-rest-server/Cargo.toml
@@ -7,3 +7,18 @@ description = "REST server implementation"
[dependencies]
anyhow = "1.0"
+futures = "0.3"
+handlebars = "3.0"
+http = "0.2"
+hyper = { version = "0.14", features = [ "full" ] }
+lazy_static = "1.4"
+libc = "0.2"
+nix = "0.19.1"
+serde = { version = "1.0", features = [] }
+serde_json = "1.0"
+tokio = { version = "1.6", features = ["signal", "process"] }
+
+proxmox = { version = "0.13.3", features = [ "router"] }
+
+# fixme: remove this dependency (pbs_tools::broadcast_future)
+pbs-tools = { path = "../pbs-tools" }
diff --git a/src/server/config.rs b/proxmox-rest-server/src/api_config.rs
similarity index 92%
rename from src/server/config.rs
rename to proxmox-rest-server/src/api_config.rs
index 195d7a88..a319e204 100644
--- a/src/server/config.rs
+++ b/proxmox-rest-server/src/api_config.rs
@@ -12,8 +12,7 @@ use serde::Serialize;
use proxmox::api::{ApiMethod, Router, RpcEnvironmentType};
use proxmox::tools::fs::{create_path, CreateOptions};
-use crate::tools::{FileLogger, FileLogOptions};
-use super::auth::ApiAuth;
+use crate::{ApiAuth, FileLogger, FileLogOptions, CommandoSocket};
pub struct ApiConfig {
basedir: PathBuf,
@@ -134,7 +133,9 @@ impl ApiConfig {
pub fn enable_file_log<P>(
&mut self,
path: P,
- commando_sock: &mut super::CommandoSocket,
+ dir_opts: Option<CreateOptions>,
+ file_opts: Option<CreateOptions>,
+ commando_sock: &mut CommandoSocket,
) -> Result<(), Error>
where
P: Into<PathBuf>
@@ -142,15 +143,13 @@ impl ApiConfig {
let path: PathBuf = path.into();
if let Some(base) = path.parent() {
if !base.exists() {
- let backup_user = pbs_config::backup_user()?;
- let opts = CreateOptions::new().owner(backup_user.uid).group(backup_user.gid);
- create_path(base, None, Some(opts)).map_err(|err| format_err!("{}", err))?;
+ create_path(base, None, dir_opts).map_err(|err| format_err!("{}", err))?;
}
}
let logger_options = FileLogOptions {
append: true,
- owned_by_backup: true,
+ file_opts: file_opts.unwrap_or(CreateOptions::default()),
..Default::default()
};
let request_log = Arc::new(Mutex::new(FileLogger::new(&path, logger_options)?));
diff --git a/src/server/command_socket.rs b/proxmox-rest-server/src/command_socket.rs
similarity index 94%
rename from src/server/command_socket.rs
rename to proxmox-rest-server/src/command_socket.rs
index e3bd0c12..1d62d21d 100644
--- a/src/server/command_socket.rs
+++ b/proxmox-rest-server/src/command_socket.rs
@@ -10,17 +10,17 @@ use tokio::net::UnixListener;
use serde::Serialize;
use serde_json::Value;
use nix::sys::socket;
+use nix::unistd::Gid;
-/// Listens on a Unix Socket to handle simple command asynchronously
-fn create_control_socket<P, F>(path: P, func: F) -> Result<impl Future<Output = ()>, Error>
+// Listens on a Unix Socket to handle simple command asynchronously
+fn create_control_socket<P, F>(path: P, gid: Gid, func: F) -> Result<impl Future<Output = ()>, Error>
where
P: Into<PathBuf>,
F: Fn(Value) -> Result<Value, Error> + Send + Sync + 'static,
{
let path: PathBuf = path.into();
- let backup_user = pbs_config::backup_user()?;
- let backup_gid = backup_user.gid.as_raw();
+ let gid = gid.as_raw();
let socket = UnixListener::bind(&path)?;
@@ -47,7 +47,7 @@ where
// check permissions (same gid, root user, or backup group)
let mygid = unsafe { libc::getgid() };
- if !(cred.uid() == 0 || cred.gid() == mygid || cred.gid() == backup_gid) {
+ if !(cred.uid() == 0 || cred.gid() == mygid || cred.gid() == gid) {
eprintln!("no permissions for {:?}", cred);
continue;
}
@@ -93,7 +93,7 @@ where
}
}.boxed();
- let abort_future = super::last_worker_future().map_err(|_| {});
+ let abort_future = crate::last_worker_future().map_err(|_| {});
let task = futures::future::select(
control_future,
abort_future,
@@ -154,15 +154,17 @@ pub type CommandoSocketFn = Box<(dyn Fn(Option<&Value>) -> Result<Value, Error>
/// You need to call `spawn()` to make the socket active.
pub struct CommandoSocket {
socket: PathBuf,
+ gid: Gid,
commands: HashMap<String, CommandoSocketFn>,
}
impl CommandoSocket {
- pub fn new<P>(path: P) -> Self
+ pub fn new<P>(path: P, gid: Gid) -> Self
where P: Into<PathBuf>,
{
CommandoSocket {
socket: path.into(),
+ gid,
commands: HashMap::new(),
}
}
@@ -170,7 +172,7 @@ impl CommandoSocket {
/// Spawn the socket and consume self, meaning you cannot register commands anymore after
/// calling this.
pub fn spawn(self) -> Result<(), Error> {
- let control_future = create_control_socket(self.socket.to_owned(), move |param| {
+ let control_future = create_control_socket(self.socket.to_owned(), self.gid, move |param| {
let param = param
.as_object()
.ok_or_else(|| format_err!("unable to parse parameters (expected json object)"))?;
diff --git a/src/tools/file_logger.rs b/proxmox-rest-server/src/file_logger.rs
similarity index 81%
rename from src/tools/file_logger.rs
rename to proxmox-rest-server/src/file_logger.rs
index 5b8db2c5..31100e40 100644
--- a/src/tools/file_logger.rs
+++ b/proxmox-rest-server/src/file_logger.rs
@@ -1,6 +1,10 @@
-use anyhow::Error;
use std::io::Write;
+use anyhow::Error;
+use nix::fcntl::OFlag;
+
+use proxmox::tools::fs::{CreateOptions, atomic_open_or_create_file};
+
/// Log messages with optional automatically added timestamps into files
///
/// Logs messages to file, and optionally to standard output.
@@ -9,8 +13,7 @@ use std::io::Write;
/// #### Example:
/// ```
/// # use anyhow::{bail, format_err, Error};
-/// use proxmox_backup::flog;
-/// use proxmox_backup::tools::{FileLogger, FileLogOptions};
+/// use proxmox_rest_server::{flog, FileLogger, FileLogOptions};
///
/// # std::fs::remove_file("test.log");
/// let options = FileLogOptions {
@@ -23,7 +26,7 @@ use std::io::Write;
/// # std::fs::remove_file("test.log");
/// ```
-#[derive(Debug, Default)]
+#[derive(Default)]
/// Options to control the behavior of a ['FileLogger'] instance
pub struct FileLogOptions {
/// Open underlying log file in append mode, useful when multiple concurrent processes
@@ -39,13 +42,11 @@ pub struct FileLogOptions {
pub to_stdout: bool,
/// Prefix messages logged to the file with the current local time as RFC 3339
pub prefix_time: bool,
- /// if set, the file is tried to be chowned by the backup:backup user/group
- /// Note, this is not designed race free as anybody could set it to another user afterwards
- /// anyway. It must thus be used by all processes which doe not run as backup uid/gid.
- pub owned_by_backup: bool,
+ /// File owner/group and mode
+ pub file_opts: CreateOptions,
+
}
-#[derive(Debug)]
pub struct FileLogger {
file: std::fs::File,
file_name: std::path::PathBuf,
@@ -82,19 +83,24 @@ impl FileLogger {
file_name: P,
options: &FileLogOptions,
) -> Result<std::fs::File, Error> {
- let file = std::fs::OpenOptions::new()
- .read(options.read)
- .write(true)
- .append(options.append)
- .create_new(options.exclusive)
- .create(!options.exclusive)
- .open(&file_name)?;
-
- if options.owned_by_backup {
- let backup_user = pbs_config::backup_user()?;
- nix::unistd::chown(file_name.as_ref(), Some(backup_user.uid), Some(backup_user.gid))?;
+
+ let mut flags = OFlag::O_CLOEXEC;
+
+ if options.read {
+ flags |= OFlag::O_RDWR;
+ } else {
+ flags |= OFlag::O_WRONLY;
+ }
+
+ if options.append {
+ flags |= OFlag::O_APPEND;
+ }
+ if options.exclusive {
+ flags |= OFlag::O_EXCL;
}
+ let file = atomic_open_or_create_file(&file_name, flags, &[], options.file_opts.clone())?;
+
Ok(file)
}
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index e69de29b..38dd610c 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -0,0 +1,54 @@
+use anyhow::{bail, Error};
+
+mod state;
+pub use state::*;
+
+mod command_socket;
+pub use command_socket::*;
+
+mod file_logger;
+pub use file_logger::{FileLogger, FileLogOptions};
+
+mod api_config;
+pub use api_config::ApiConfig;
+
+pub enum AuthError {
+ Generic(Error),
+ NoData,
+}
+
+impl From<Error> for AuthError {
+ fn from(err: Error) -> Self {
+ AuthError::Generic(err)
+ }
+}
+
+pub trait ApiAuth {
+ fn check_auth(
+ &self,
+ headers: &http::HeaderMap,
+ method: &hyper::Method,
+ ) -> Result<String, AuthError>;
+}
+
+static mut SHUTDOWN_REQUESTED: bool = false;
+
+pub fn request_shutdown() {
+ unsafe {
+ SHUTDOWN_REQUESTED = true;
+ }
+ crate::server_shutdown();
+}
+
+#[inline(always)]
+pub fn shutdown_requested() -> bool {
+ unsafe { SHUTDOWN_REQUESTED }
+}
+
+pub fn fail_on_shutdown() -> Result<(), Error> {
+ if shutdown_requested() {
+ bail!("Server shutdown requested - aborting task");
+ }
+ Ok(())
+}
+
diff --git a/src/server/state.rs b/proxmox-rest-server/src/state.rs
similarity index 97%
rename from src/server/state.rs
rename to proxmox-rest-server/src/state.rs
index d294c935..468ef0aa 100644
--- a/src/server/state.rs
+++ b/proxmox-rest-server/src/state.rs
@@ -42,7 +42,7 @@ pub fn server_state_init() -> Result<(), Error> {
while stream.recv().await.is_some() {
println!("got shutdown request (SIGINT)");
SERVER_STATE.lock().unwrap().reload_request = false;
- crate::tools::request_shutdown();
+ crate::request_shutdown();
}
}.boxed();
@@ -57,7 +57,7 @@ pub fn server_state_init() -> Result<(), Error> {
while stream.recv().await.is_some() {
println!("got reload request (SIGHUP)");
SERVER_STATE.lock().unwrap().reload_request = true;
- crate::tools::request_shutdown();
+ crate::request_shutdown();
}
}.boxed();
diff --git a/pxar-bin/Cargo.toml b/pxar-bin/Cargo.toml
index e1a47604..e322e654 100644
--- a/pxar-bin/Cargo.toml
+++ b/pxar-bin/Cargo.toml
@@ -16,7 +16,7 @@ serde_json = "1.0"
tokio = { version = "1.6", features = [ "rt", "rt-multi-thread" ] }
pathpatterns = "0.1.2"
-proxmox = { version = "0.13.0", default-features = false, features = [] }
+proxmox = { version = "0.13.3", default-features = false, features = [] }
pxar = { version = "0.10.1", features = [ "tokio-io" ] }
pbs-client = { path = "../pbs-client" }
diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index 33700a90..dc1c0ae9 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -1505,7 +1505,7 @@ pub fn pxar_file_download(
EntryKind::Directory => {
let (sender, receiver) = tokio::sync::mpsc::channel(100);
let channelwriter = AsyncChannelWriter::new(sender, 1024 * 1024);
- crate::server::spawn_internal_task(
+ proxmox_rest_server::spawn_internal_task(
create_zip(channelwriter, decoder, path.clone(), false)
);
Body::wrap_stream(ReceiverStream::new(receiver).map_err(move |err| {
diff --git a/src/api2/node/mod.rs b/src/api2/node/mod.rs
index 898e6291..ecc1e2e0 100644
--- a/src/api2/node/mod.rs
+++ b/src/api2/node/mod.rs
@@ -295,12 +295,12 @@ fn upgrade_to_websocket(
let (ws, response) = WebSocket::new(parts.headers.clone())?;
- crate::server::spawn_internal_task(async move {
+ proxmox_rest_server::spawn_internal_task(async move {
let conn: Upgraded = match hyper::upgrade::on(Request::from_parts(parts, req_body))
.map_err(Error::from)
.await
{
- Ok(upgraded) => upgraded,
+ Ok(upgraded) => upgraded,
_ => bail!("error"),
};
diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs
index d248ecaf..df8d46b6 100644
--- a/src/backup/datastore.rs
+++ b/src/backup/datastore.rs
@@ -29,8 +29,7 @@ use pbs_tools::format::HumanByte;
use pbs_tools::fs::{lock_dir_noblock, DirLockGuard};
use pbs_tools::process_locker::ProcessLockSharedGuard;
use pbs_config::{open_backup_lockfile, BackupLockGuard};
-
-use crate::tools::fail_on_shutdown;
+use proxmox_rest_server::fail_on_shutdown;
lazy_static! {
static ref DATASTORE_MAP: Mutex<HashMap<String, Arc<DataStore>>> = Mutex::new(HashMap::new());
diff --git a/src/backup/verify.rs b/src/backup/verify.rs
index 6e188c5f..b8d2b2f3 100644
--- a/src/backup/verify.rs
+++ b/src/backup/verify.rs
@@ -172,7 +172,7 @@ fn verify_index_chunks(
let check_abort = |pos: usize| -> Result<(), Error> {
if pos & 1023 == 0 {
verify_worker.worker.check_abort()?;
- crate::tools::fail_on_shutdown()?;
+ proxmox_rest_server::fail_on_shutdown()?;
}
Ok(())
};
@@ -184,7 +184,7 @@ fn verify_index_chunks(
for (pos, _) in chunk_list {
verify_worker.worker.check_abort()?;
- crate::tools::fail_on_shutdown()?;
+ proxmox_rest_server::fail_on_shutdown()?;
let info = index.chunk_info(pos).unwrap();
@@ -376,7 +376,7 @@ pub fn verify_backup_dir_with_lock(
});
verify_worker.worker.check_abort()?;
- crate::tools::fail_on_shutdown()?;
+ proxmox_rest_server::fail_on_shutdown()?;
if let Err(err) = result {
task_log!(
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index c8751bc5..452bbc3a 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -3,8 +3,10 @@ use futures::*;
use proxmox::try_block;
use proxmox::api::RpcEnvironmentType;
+use proxmox::tools::fs::CreateOptions;
use pbs_tools::auth::private_auth_key;
+use proxmox_rest_server::ApiConfig;
use proxmox_backup::server::{
self,
@@ -57,16 +59,25 @@ async fn run() -> Result<(), Error> {
}
let _ = csrf_secret(); // load with lazy_static
- let mut config = server::ApiConfig::new(
+ let mut config = ApiConfig::new(
pbs_buildcfg::JS_DIR,
&proxmox_backup::api2::ROUTER,
RpcEnvironmentType::PRIVILEGED,
default_api_auth(),
)?;
- let mut commando_sock = server::CommandoSocket::new(server::our_ctrl_sock());
+ let backup_user = pbs_config::backup_user()?;
+ let mut commando_sock = proxmox_rest_server::CommandoSocket::new(crate::server::our_ctrl_sock(), backup_user.gid);
- config.enable_file_log(pbs_buildcfg::API_ACCESS_LOG_FN, &mut commando_sock)?;
+ let dir_opts = CreateOptions::new().owner(backup_user.uid).group(backup_user.gid);
+ let file_opts = CreateOptions::new().owner(backup_user.uid).group(backup_user.gid);
+
+ config.enable_file_log(
+ pbs_buildcfg::API_ACCESS_LOG_FN,
+ Some(dir_opts),
+ Some(file_opts),
+ &mut commando_sock,
+ )?;
let rest_server = RestServer::new(config);
@@ -78,7 +89,7 @@ async fn run() -> Result<(), Error> {
Ok(ready
.and_then(|_| hyper::Server::builder(incoming)
.serve(rest_server)
- .with_graceful_shutdown(server::shutdown_future())
+ .with_graceful_shutdown(proxmox_rest_server::shutdown_future())
.map_err(Error::from)
)
.map(|e| {
@@ -97,7 +108,7 @@ async fn run() -> Result<(), Error> {
let init_result: Result<(), Error> = try_block!({
server::register_task_control_commands(&mut commando_sock)?;
commando_sock.spawn()?;
- server::server_state_init()?;
+ proxmox_rest_server::server_state_init()?;
Ok(())
});
@@ -107,7 +118,7 @@ async fn run() -> Result<(), Error> {
server.await?;
log::info!("server shutting down, waiting for active workers to complete");
- proxmox_backup::server::last_worker_future().await?;
+ proxmox_rest_server::last_worker_future().await?;
log::info!("done - exit server");
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 4240711f..de534a65 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -12,13 +12,15 @@ use serde_json::Value;
use proxmox::try_block;
use proxmox::api::RpcEnvironmentType;
use proxmox::sys::linux::socket::set_tcp_keepalive;
+use proxmox::tools::fs::CreateOptions;
+
+use proxmox_rest_server::ApiConfig;
use proxmox_backup::{
backup::DataStore,
server::{
auth::default_api_auth,
WorkerTask,
- ApiConfig,
rest::*,
jobstate::{
self,
@@ -106,9 +108,18 @@ async fn run() -> Result<(), Error> {
config.register_template("index", &indexpath)?;
config.register_template("console", "/usr/share/pve-xtermjs/index.html.hbs")?;
- let mut commando_sock = server::CommandoSocket::new(server::our_ctrl_sock());
+ let backup_user = pbs_config::backup_user()?;
+ let mut commando_sock = proxmox_rest_server::CommandoSocket::new(crate::server::our_ctrl_sock(), backup_user.gid);
+
+ let dir_opts = CreateOptions::new().owner(backup_user.uid).group(backup_user.gid);
+ let file_opts = CreateOptions::new().owner(backup_user.uid).group(backup_user.gid);
- config.enable_file_log(pbs_buildcfg::API_ACCESS_LOG_FN, &mut commando_sock)?;
+ config.enable_file_log(
+ pbs_buildcfg::API_ACCESS_LOG_FN,
+ Some(dir_opts),
+ Some(file_opts),
+ &mut commando_sock,
+ )?;
let rest_server = RestServer::new(config);
@@ -158,7 +169,7 @@ async fn run() -> Result<(), Error> {
Ok(ready
.and_then(|_| hyper::Server::builder(connections)
.serve(rest_server)
- .with_graceful_shutdown(server::shutdown_future())
+ .with_graceful_shutdown(proxmox_rest_server::shutdown_future())
.map_err(Error::from)
)
.map_err(|err| eprintln!("server error: {}", err))
@@ -174,7 +185,7 @@ async fn run() -> Result<(), Error> {
let init_result: Result<(), Error> = try_block!({
server::register_task_control_commands(&mut commando_sock)?;
commando_sock.spawn()?;
- server::server_state_init()?;
+ proxmox_rest_server::server_state_init()?;
Ok(())
});
@@ -187,7 +198,7 @@ async fn run() -> Result<(), Error> {
server.await?;
log::info!("server shutting down, waiting for active workers to complete");
- proxmox_backup::server::last_worker_future().await?;
+ proxmox_rest_server::last_worker_future().await?;
log::info!("done - exit server");
Ok(())
@@ -304,14 +315,14 @@ async fn accept_connection(
}
fn start_stat_generator() {
- let abort_future = server::shutdown_future();
+ let abort_future = proxmox_rest_server::shutdown_future();
let future = Box::pin(run_stat_generator());
let task = futures::future::select(future, abort_future);
tokio::spawn(task.map(|_| ()));
}
fn start_task_scheduler() {
- let abort_future = server::shutdown_future();
+ let abort_future = proxmox_rest_server::shutdown_future();
let future = Box::pin(run_task_scheduler());
let task = futures::future::select(future, abort_future);
tokio::spawn(task.map(|_| ()));
@@ -706,12 +717,12 @@ async fn schedule_task_log_rotate() {
async fn command_reopen_logfiles() -> Result<(), Error> {
// only care about the most recent daemon instance for each, proxy & api, as other older ones
// should not respond to new requests anyway, but only finish their current one and then exit.
- let sock = server::our_ctrl_sock();
- let f1 = server::send_command(sock, "{\"command\":\"api-access-log-reopen\"}\n");
+ let sock = crate::server::our_ctrl_sock();
+ let f1 = proxmox_rest_server::send_command(sock, "{\"command\":\"api-access-log-reopen\"}\n");
- let pid = server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_API_PID_FN)?;
- let sock = server::ctrl_sock_from_pid(pid);
- let f2 = server::send_command(sock, "{\"command\":\"api-access-log-reopen\"}\n");
+ let pid = crate::server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_API_PID_FN)?;
+ let sock = crate::server::ctrl_sock_from_pid(pid);
+ let f2 = proxmox_rest_server::send_command(sock, "{\"command\":\"api-access-log-reopen\"}\n");
match futures::join!(f1, f2) {
(Err(e1), Err(e2)) => Err(format_err!("reopen commands failed, proxy: {}; api: {}", e1, e2)),
diff --git a/src/bin/proxmox-restore-daemon.rs b/src/bin/proxmox-restore-daemon.rs
index e9018ecc..c5122cff 100644
--- a/src/bin/proxmox-restore-daemon.rs
+++ b/src/bin/proxmox-restore-daemon.rs
@@ -15,9 +15,11 @@ use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
use proxmox::api::RpcEnvironmentType;
-use proxmox_backup::server::{rest::*, ApiConfig};
use pbs_client::DEFAULT_VSOCK_PORT;
+use proxmox_rest_server::ApiConfig;
+
+use proxmox_backup::server::rest::*;
mod proxmox_restore_daemon;
use proxmox_restore_daemon::*;
diff --git a/src/bin/proxmox_restore_daemon/auth.rs b/src/bin/proxmox_restore_daemon/auth.rs
index 30309bb8..ea1dabe6 100644
--- a/src/bin/proxmox_restore_daemon/auth.rs
+++ b/src/bin/proxmox_restore_daemon/auth.rs
@@ -4,10 +4,7 @@ use std::io::prelude::*;
use anyhow::{bail, format_err, Error};
-use pbs_api_types::Authid;
-
-use pbs_config::CachedUserInfo;
-use proxmox_backup::server::auth::{ApiAuth, AuthError};
+use proxmox_rest_server::{ApiAuth, AuthError};
const TICKET_FILE: &str = "/ticket";
@@ -20,11 +17,10 @@ impl ApiAuth for StaticAuth {
&self,
headers: &http::HeaderMap,
_method: &hyper::Method,
- _user_info: &CachedUserInfo,
- ) -> Result<Authid, AuthError> {
+ ) -> Result<String, AuthError> {
match headers.get(hyper::header::AUTHORIZATION) {
Some(header) if header.to_str().unwrap_or("") == &self.ticket => {
- Ok(Authid::root_auth_id().to_owned())
+ Ok(String::from("root@pam"))
}
_ => {
return Err(AuthError::Generic(format_err!(
diff --git a/src/server/auth.rs b/src/server/auth.rs
index 19933177..321d1e02 100644
--- a/src/server/auth.rs
+++ b/src/server/auth.rs
@@ -1,11 +1,12 @@
//! Provides authentication primitives for the HTTP server
-use anyhow::{format_err, Error};
+use anyhow::format_err;
use std::sync::Arc;
use pbs_tools::ticket::{self, Ticket};
use pbs_config::{token_shadow, CachedUserInfo};
use pbs_api_types::{Authid, Userid};
+use proxmox_rest_server::{ApiAuth, AuthError};
use crate::auth_helpers::*;
use crate::tools;
@@ -13,26 +14,6 @@ use crate::tools;
use hyper::header;
use percent_encoding::percent_decode_str;
-pub enum AuthError {
- Generic(Error),
- NoData,
-}
-
-impl From<Error> for AuthError {
- fn from(err: Error) -> Self {
- AuthError::Generic(err)
- }
-}
-
-pub trait ApiAuth {
- fn check_auth(
- &self,
- headers: &http::HeaderMap,
- method: &hyper::Method,
- user_info: &CachedUserInfo,
- ) -> Result<Authid, AuthError>;
-}
-
struct UserAuthData {
ticket: String,
csrf_token: Option<String>,
@@ -80,8 +61,10 @@ impl ApiAuth for UserApiAuth {
&self,
headers: &http::HeaderMap,
method: &hyper::Method,
- user_info: &CachedUserInfo,
- ) -> Result<Authid, AuthError> {
+ ) -> Result<String, AuthError> {
+
+ let user_info = CachedUserInfo::new()?;
+
let auth_data = Self::extract_auth_data(headers);
match auth_data {
Some(AuthData::User(user_auth_data)) => {
@@ -111,7 +94,7 @@ impl ApiAuth for UserApiAuth {
}
}
- Ok(auth_id)
+ Ok(auth_id.to_string())
}
Some(AuthData::ApiToken(api_token)) => {
let mut parts = api_token.splitn(2, ':');
@@ -133,7 +116,7 @@ impl ApiAuth for UserApiAuth {
token_shadow::verify_secret(&tokenid, &tokensecret)?;
- Ok(tokenid)
+ Ok(tokenid.to_string())
}
None => Err(AuthError::NoData),
}
diff --git a/src/server/mod.rs b/src/server/mod.rs
index 52c6e7bc..69cfe7b0 100644
--- a/src/server/mod.rs
+++ b/src/server/mod.rs
@@ -52,21 +52,12 @@ pub use environment::*;
mod upid;
pub use upid::*;
-mod state;
-pub use state::*;
-
-mod command_socket;
-pub use command_socket::*;
-
mod worker_task;
pub use worker_task::*;
mod h2service;
pub use h2service::*;
-pub mod config;
-pub use config::*;
-
pub mod formatter;
#[macro_use]
@@ -98,7 +89,7 @@ pub mod pull;
pub(crate) async fn reload_proxy_certificate() -> Result<(), Error> {
let proxy_pid = crate::server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_PROXY_PID_FN)?;
let sock = crate::server::ctrl_sock_from_pid(proxy_pid);
- let _: Value = crate::server::send_raw_command(sock, "{\"command\":\"reload-certificate\"}\n")
+ let _: Value = proxmox_rest_server::send_raw_command(sock, "{\"command\":\"reload-certificate\"}\n")
.await?;
Ok(())
}
@@ -106,7 +97,7 @@ pub(crate) async fn reload_proxy_certificate() -> Result<(), Error> {
pub(crate) async fn notify_datastore_removed() -> Result<(), Error> {
let proxy_pid = crate::server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_PROXY_PID_FN)?;
let sock = crate::server::ctrl_sock_from_pid(proxy_pid);
- let _: Value = crate::server::send_raw_command(sock, "{\"command\":\"datastore-removed\"}\n")
+ let _: Value = proxmox_rest_server::send_raw_command(sock, "{\"command\":\"datastore-removed\"}\n")
.await?;
Ok(())
}
diff --git a/src/server/rest.rs b/src/server/rest.rs
index a648832a..0c4df906 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -29,21 +29,20 @@ use proxmox::api::{
RpcEnvironmentType,
};
use proxmox::http_err;
+use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
use pbs_api_types::{Authid, Userid};
+use proxmox_rest_server::{ApiConfig, FileLogger, FileLogOptions, AuthError};
-use super::auth::AuthError;
use super::environment::RestEnvironment;
use super::formatter::*;
-use super::ApiConfig;
use crate::auth_helpers::*;
use pbs_config::CachedUserInfo;
use crate::tools;
use crate::tools::compression::CompressionMethod;
-use crate::tools::FileLogger;
extern "C" {
fn tzset();
@@ -196,10 +195,16 @@ fn log_response(
}
}
pub fn auth_logger() -> Result<FileLogger, Error> {
- let logger_options = tools::FileLogOptions {
+ let backup_user = pbs_config::backup_user()?;
+
+ let file_opts = CreateOptions::new()
+ .owner(backup_user.uid)
+ .group(backup_user.gid);
+
+ let logger_options = FileLogOptions {
append: true,
prefix_time: true,
- owned_by_backup: true,
+ file_opts,
..Default::default()
};
FileLogger::new(pbs_buildcfg::API_AUTH_LOG_FN, logger_options)
@@ -681,7 +686,6 @@ async fn handle_request(
rpcenv.set_client_ip(Some(*peer));
- let user_info = CachedUserInfo::new()?;
let auth = &api.api_auth;
let delay_unauth_time = std::time::Instant::now() + std::time::Duration::from_millis(3000);
@@ -708,8 +712,8 @@ async fn handle_request(
}
if auth_required {
- match auth.check_auth(&parts.headers, &method, &user_info) {
- Ok(authid) => rpcenv.set_auth_id(Some(authid.to_string())),
+ match auth.check_auth(&parts.headers, &method) {
+ Ok(authid) => rpcenv.set_auth_id(Some(authid)),
Err(auth_err) => {
let err = match auth_err {
AuthError::Generic(err) => err,
@@ -738,6 +742,8 @@ async fn handle_request(
}
Some(api_method) => {
let auth_id = rpcenv.get_auth_id();
+ let user_info = CachedUserInfo::new()?;
+
if !check_api_permission(
api_method.access.permission,
auth_id.as_deref(),
@@ -779,8 +785,9 @@ async fn handle_request(
if comp_len == 0 {
let language = extract_lang_header(&parts.headers);
- match auth.check_auth(&parts.headers, &method, &user_info) {
+ match auth.check_auth(&parts.headers, &method) {
Ok(auth_id) => {
+ let auth_id: Authid = auth_id.parse()?;
if !auth_id.is_token() {
let userid = auth_id.user();
let new_csrf_token = assemble_csrf_prevention_token(csrf_secret(), userid);
diff --git a/src/server/worker_task.rs b/src/server/worker_task.rs
index 2ef8ba9d..c62ba7df 100644
--- a/src/server/worker_task.rs
+++ b/src/server/worker_task.rs
@@ -20,12 +20,10 @@ use pbs_buildcfg;
use pbs_tools::logrotate::{LogRotate, LogRotateFiles};
use pbs_api_types::{Authid, TaskStateType, UPID};
use pbs_config::{open_backup_lockfile, BackupLockGuard};
+use proxmox_rest_server::{CommandoSocket, FileLogger, FileLogOptions};
use super::UPIDExt;
-use crate::server;
-use crate::tools::{FileLogger, FileLogOptions};
-
macro_rules! taskdir {
($subdir:expr) => (concat!(pbs_buildcfg::PROXMOX_BACKUP_LOG_DIR_M!(), "/tasks", $subdir))
}
@@ -41,7 +39,7 @@ lazy_static! {
/// checks if the task UPID refers to a worker from this process
fn is_local_worker(upid: &UPID) -> bool {
- upid.pid == server::pid() && upid.pstart == server::pstart()
+ upid.pid == crate::server::pid() && upid.pstart == crate::server::pstart()
}
/// Test if the task is still running
@@ -54,14 +52,14 @@ pub async fn worker_is_active(upid: &UPID) -> Result<bool, Error> {
return Ok(false);
}
- let sock = server::ctrl_sock_from_pid(upid.pid);
+ let sock = crate::server::ctrl_sock_from_pid(upid.pid);
let cmd = json!({
"command": "worker-task-status",
"args": {
"upid": upid.to_string(),
},
});
- let status = super::send_command(sock, &cmd).await?;
+ let status = proxmox_rest_server::send_command(sock, &cmd).await?;
if let Some(active) = status.as_bool() {
Ok(active)
@@ -84,7 +82,7 @@ pub fn worker_is_active_local(upid: &UPID) -> bool {
}
pub fn register_task_control_commands(
- commando_sock: &mut super::CommandoSocket,
+ commando_sock: &mut CommandoSocket,
) -> Result<(), Error> {
fn get_upid(args: Option<&Value>) -> Result<UPID, Error> {
let args = if let Some(args) = args { args } else { bail!("missing args") };
@@ -128,14 +126,14 @@ pub fn abort_worker_async(upid: UPID) {
pub async fn abort_worker(upid: UPID) -> Result<(), Error> {
- let sock = server::ctrl_sock_from_pid(upid.pid);
+ let sock = crate::server::ctrl_sock_from_pid(upid.pid);
let cmd = json!({
"command": "worker-task-abort",
"args": {
"upid": upid.to_string(),
},
});
- super::send_command(sock, &cmd).map_ok(|_| ()).await
+ proxmox_rest_server::send_command(sock, &cmd).map_ok(|_| ()).await
}
fn parse_worker_status_line(line: &str) -> Result<(String, UPID, Option<TaskState>), Error> {
@@ -579,7 +577,6 @@ impl Iterator for TaskListInfoIterator {
/// task/future. Each task can `log()` messages, which are stored
/// persistently to files. Task should poll the `abort_requested`
/// flag, and stop execution when requested.
-#[derive(Debug)]
pub struct WorkerTask {
upid: UPID,
data: Mutex<WorkerTaskData>,
@@ -593,7 +590,6 @@ impl std::fmt::Display for WorkerTask {
}
}
-#[derive(Debug)]
struct WorkerTaskData {
logger: FileLogger,
progress: f64, // 0..1
@@ -642,7 +638,7 @@ impl WorkerTask {
{
let mut hash = WORKER_TASK_LIST.lock().unwrap();
hash.insert(task_id, worker.clone());
- super::set_worker_count(hash.len());
+ proxmox_rest_server::set_worker_count(hash.len());
}
update_active_workers(Some(&upid))?;
@@ -729,7 +725,7 @@ impl WorkerTask {
WORKER_TASK_LIST.lock().unwrap().remove(&self.upid.task_id);
let _ = update_active_workers(None);
- super::set_worker_count(WORKER_TASK_LIST.lock().unwrap().len());
+ proxmox_rest_server::set_worker_count(WORKER_TASK_LIST.lock().unwrap().len());
}
/// Log a message.
diff --git a/src/tools/daemon.rs b/src/tools/daemon.rs
index d298bf16..1291601b 100644
--- a/src/tools/daemon.rs
+++ b/src/tools/daemon.rs
@@ -16,7 +16,6 @@ use futures::future::{self, Either};
use proxmox::tools::io::{ReadExt, WriteExt};
-use crate::server;
use crate::tools::{fd_change_cloexec, self};
#[link(name = "systemd")]
@@ -274,11 +273,11 @@ where
).await?;
let server_future = create_service(listener, NotifyReady)?;
- let shutdown_future = server::shutdown_future();
+ let shutdown_future = proxmox_rest_server::shutdown_future();
let finish_future = match future::select(server_future, shutdown_future).await {
Either::Left((_, _)) => {
- crate::tools::request_shutdown(); // make sure we are in shutdown mode
+ proxmox_rest_server::request_shutdown(); // make sure we are in shutdown mode
None
}
Either::Right((_, server_future)) => Some(server_future),
@@ -286,7 +285,7 @@ where
let mut reloader = Some(reloader);
- if server::is_reload_request() {
+ if proxmox_rest_server::is_reload_request() {
log::info!("daemon reload...");
if let Err(e) = systemd_notify(SystemdNotify::Reloading) {
log::error!("failed to notify systemd about the state change: {}", e);
@@ -305,7 +304,7 @@ where
}
// FIXME: this is a hack, replace with sd_notify_barrier when available
- if server::is_reload_request() {
+ if proxmox_rest_server::is_reload_request() {
wait_service_is_not_state(service_name, "reloading").await?;
}
diff --git a/src/tools/mod.rs b/src/tools/mod.rs
index 64e592b2..f8b363f5 100644
--- a/src/tools/mod.rs
+++ b/src/tools/mod.rs
@@ -31,9 +31,6 @@ pub mod ticket;
pub mod parallel_handler;
pub use parallel_handler::ParallelHandler;
-mod file_logger;
-pub use file_logger::{FileLogger, FileLogOptions};
-
/// Shortcut for md5 sums.
pub fn md5sum(data: &[u8]) -> Result<DigestBytes, Error> {
hash(MessageDigest::md5(), data).map_err(Error::from)
@@ -123,27 +120,6 @@ pub fn fd_change_cloexec(fd: RawFd, on: bool) -> Result<(), Error> {
Ok(())
}
-static mut SHUTDOWN_REQUESTED: bool = false;
-
-pub fn request_shutdown() {
- unsafe {
- SHUTDOWN_REQUESTED = true;
- }
- crate::server::server_shutdown();
-}
-
-#[inline(always)]
-pub fn shutdown_requested() -> bool {
- unsafe { SHUTDOWN_REQUESTED }
-}
-
-pub fn fail_on_shutdown() -> Result<(), Error> {
- if shutdown_requested() {
- bail!("Server shutdown requested - aborting task");
- }
- Ok(())
-}
-
/// safe wrapper for `nix::sys::socket::socketpair` defaulting to `O_CLOEXEC` and guarding the file
/// descriptors.
pub fn socketpair() -> Result<(Fd, Fd), Error> {
diff --git a/tests/worker-task-abort.rs b/tests/worker-task-abort.rs
index 736ae659..7271ea55 100644
--- a/tests/worker-task-abort.rs
+++ b/tests/worker-task-abort.rs
@@ -1,6 +1,5 @@
use anyhow::{bail, Error};
-#[macro_use]
extern crate proxmox_backup;
extern crate tokio;
@@ -10,8 +9,8 @@ use proxmox::try_block;
use pbs_api_types::{Authid, UPID};
+use proxmox_rest_server::{flog, CommandoSocket};
use proxmox_backup::server;
-use proxmox_backup::tools;
fn garbage_collection(worker: &server::WorkerTask) -> Result<(), Error> {
@@ -45,11 +44,11 @@ fn worker_task_abort() -> Result<(), Error> {
let rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async move {
- let mut commando_sock = server::CommandoSocket::new(server::our_ctrl_sock());
+ let mut commando_sock = CommandoSocket::new(server::our_ctrl_sock(), nix::unistd::Gid::current());
let init_result: Result<(), Error> = try_block!({
server::register_task_control_commands(&mut commando_sock)?;
- server::server_state_init()?;
+ proxmox_rest_server::server_state_init()?;
Ok(())
});
@@ -73,7 +72,7 @@ fn worker_task_abort() -> Result<(), Error> {
println!("WORKER {}", worker);
let result = garbage_collection(&worker);
- tools::request_shutdown();
+ proxmox_rest_server::request_shutdown();
if let Err(err) = result {
println!("got expected error: {}", err);
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 03/16] move src/tools/daemon.rs to proxmox-rest-server workspace
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 01/16] start new proxmox-rest-server workspace Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 02/16] move ApiConfig, FileLogger and CommandoSocket to " Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 04/16] move src/server/environment.rs to proxmox-rest-server crate Dietmar Maurer
` (13 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
proxmox-rest-server/Cargo.toml | 1 +
.../src}/daemon.rs | 13 ++++----
proxmox-rest-server/src/lib.rs | 31 ++++++++++++++++++-
src/api2/node/mod.rs | 2 +-
src/bin/proxmox-backup-api.rs | 3 +-
src/bin/proxmox-backup-proxy.rs | 3 +-
src/tools/mod.rs | 27 ----------------
7 files changed, 43 insertions(+), 37 deletions(-)
rename {src/tools => proxmox-rest-server/src}/daemon.rs (97%)
diff --git a/proxmox-rest-server/Cargo.toml b/proxmox-rest-server/Cargo.toml
index 33ed6f39..4d1f1459 100644
--- a/proxmox-rest-server/Cargo.toml
+++ b/proxmox-rest-server/Cargo.toml
@@ -13,6 +13,7 @@ http = "0.2"
hyper = { version = "0.14", features = [ "full" ] }
lazy_static = "1.4"
libc = "0.2"
+log = "0.4"
nix = "0.19.1"
serde = { version = "1.0", features = [] }
serde_json = "1.0"
diff --git a/src/tools/daemon.rs b/proxmox-rest-server/src/daemon.rs
similarity index 97%
rename from src/tools/daemon.rs
rename to proxmox-rest-server/src/daemon.rs
index 1291601b..5401e30c 100644
--- a/src/tools/daemon.rs
+++ b/proxmox-rest-server/src/daemon.rs
@@ -15,8 +15,9 @@ use anyhow::{bail, format_err, Error};
use futures::future::{self, Either};
use proxmox::tools::io::{ReadExt, WriteExt};
+use proxmox::tools::fd::Fd;
-use crate::tools::{fd_change_cloexec, self};
+use crate::fd_change_cloexec;
#[link(name = "systemd")]
extern "C" {
@@ -218,7 +219,7 @@ impl Reloadable for tokio::net::TcpListener {
// FIXME: We could become "independent" of the TcpListener and its reference to the file
// descriptor by `dup()`ing it (and check if the listener still exists via kcmp()?)
fn get_store_func(&self) -> Result<BoxedStoreFunc, Error> {
- let mut fd_opt = Some(tools::Fd(
+ let mut fd_opt = Some(Fd(
nix::fcntl::fcntl(self.as_raw_fd(), nix::fcntl::FcntlArg::F_DUPFD_CLOEXEC(0))?
));
Ok(Box::new(move || {
@@ -273,11 +274,11 @@ where
).await?;
let server_future = create_service(listener, NotifyReady)?;
- let shutdown_future = proxmox_rest_server::shutdown_future();
+ let shutdown_future = crate::shutdown_future();
let finish_future = match future::select(server_future, shutdown_future).await {
Either::Left((_, _)) => {
- proxmox_rest_server::request_shutdown(); // make sure we are in shutdown mode
+ crate::request_shutdown(); // make sure we are in shutdown mode
None
}
Either::Right((_, server_future)) => Some(server_future),
@@ -285,7 +286,7 @@ where
let mut reloader = Some(reloader);
- if proxmox_rest_server::is_reload_request() {
+ if crate::is_reload_request() {
log::info!("daemon reload...");
if let Err(e) = systemd_notify(SystemdNotify::Reloading) {
log::error!("failed to notify systemd about the state change: {}", e);
@@ -304,7 +305,7 @@ where
}
// FIXME: this is a hack, replace with sd_notify_barrier when available
- if proxmox_rest_server::is_reload_request() {
+ if crate::is_reload_request() {
wait_service_is_not_state(service_name, "reloading").await?;
}
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 38dd610c..21a91115 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -1,4 +1,10 @@
-use anyhow::{bail, Error};
+use std::os::unix::io::RawFd;
+
+use anyhow::{bail, format_err, Error};
+
+use proxmox::tools::fd::Fd;
+
+pub mod daemon;
mod state;
pub use state::*;
@@ -52,3 +58,26 @@ pub fn fail_on_shutdown() -> Result<(), Error> {
Ok(())
}
+/// Helper to set/clear the FD_CLOEXEC flag on file descriptors
+pub fn fd_change_cloexec(fd: RawFd, on: bool) -> Result<(), Error> {
+ use nix::fcntl::{fcntl, FdFlag, F_GETFD, F_SETFD};
+ let mut flags = FdFlag::from_bits(fcntl(fd, F_GETFD)?)
+ .ok_or_else(|| format_err!("unhandled file flags"))?; // nix crate is stupid this way...
+ flags.set(FdFlag::FD_CLOEXEC, on);
+ fcntl(fd, F_SETFD(flags))?;
+ Ok(())
+}
+
+/// safe wrapper for `nix::sys::socket::socketpair` defaulting to `O_CLOEXEC` and guarding the file
+/// descriptors.
+pub fn socketpair() -> Result<(Fd, Fd), Error> {
+ use nix::sys::socket;
+ let (pa, pb) = socket::socketpair(
+ socket::AddressFamily::Unix,
+ socket::SockType::Stream,
+ None,
+ socket::SockFlag::SOCK_CLOEXEC,
+ )?;
+ Ok((Fd(pa), Fd(pb)))
+}
+
diff --git a/src/api2/node/mod.rs b/src/api2/node/mod.rs
index ecc1e2e0..9b31d595 100644
--- a/src/api2/node/mod.rs
+++ b/src/api2/node/mod.rs
@@ -151,7 +151,7 @@ async fn termproxy(cmd: Option<String>, rpcenv: &mut dyn RpcEnvironment) -> Resu
move |worker| async move {
// move inside the worker so that it survives and does not close the port
// remove CLOEXEC from listenere so that we can reuse it in termproxy
- tools::fd_change_cloexec(listener.as_raw_fd(), false)?;
+ proxmox_rest_server::fd_change_cloexec(listener.as_raw_fd(), false)?;
let mut arguments: Vec<&str> = Vec::new();
let fd_string = listener.as_raw_fd().to_string();
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index 452bbc3a..17b6f184 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -13,7 +13,8 @@ use proxmox_backup::server::{
auth::default_api_auth,
rest::*,
};
-use proxmox_backup::tools::daemon;
+use proxmox_rest_server::daemon;
+
use proxmox_backup::auth_helpers::*;
use proxmox_backup::config;
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index de534a65..d4ac2a85 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -39,11 +39,12 @@ use pbs_api_types::{
PruneOptions,
};
+use proxmox_rest_server::daemon;
+
use proxmox_backup::server;
use proxmox_backup::auth_helpers::*;
use proxmox_backup::tools::{
PROXMOX_BACKUP_TCP_KEEPALIVE_TIME,
- daemon,
disks::{
DiskManage,
zfs_pool_stats,
diff --git a/src/tools/mod.rs b/src/tools/mod.rs
index f8b363f5..8fd441b5 100644
--- a/src/tools/mod.rs
+++ b/src/tools/mod.rs
@@ -2,13 +2,10 @@
//!
//! This is a collection of small and useful tools.
use std::any::Any;
-use std::os::unix::io::RawFd;
use anyhow::{bail, format_err, Error};
use openssl::hash::{hash, DigestBytes, MessageDigest};
-pub use proxmox::tools::fd::Fd;
-
use proxmox_http::{
client::SimpleHttp,
client::SimpleHttpOptions,
@@ -19,7 +16,6 @@ pub mod apt;
pub mod async_io;
pub mod compression;
pub mod config;
-pub mod daemon;
pub mod disks;
pub mod serde_filter;
@@ -111,29 +107,6 @@ pub fn normalize_uri_path(path: &str) -> Result<(String, Vec<&str>), Error> {
Ok((path, components))
}
-pub fn fd_change_cloexec(fd: RawFd, on: bool) -> Result<(), Error> {
- use nix::fcntl::{fcntl, FdFlag, F_GETFD, F_SETFD};
- let mut flags = FdFlag::from_bits(fcntl(fd, F_GETFD)?)
- .ok_or_else(|| format_err!("unhandled file flags"))?; // nix crate is stupid this way...
- flags.set(FdFlag::FD_CLOEXEC, on);
- fcntl(fd, F_SETFD(flags))?;
- Ok(())
-}
-
-/// safe wrapper for `nix::sys::socket::socketpair` defaulting to `O_CLOEXEC` and guarding the file
-/// descriptors.
-pub fn socketpair() -> Result<(Fd, Fd), Error> {
- use nix::sys::socket;
- let (pa, pb) = socket::socketpair(
- socket::AddressFamily::Unix,
- socket::SockType::Stream,
- None,
- socket::SockFlag::SOCK_CLOEXEC,
- )?;
- Ok((Fd(pa), Fd(pb)))
-}
-
-
/// An easy way to convert types to Any
///
/// Mostly useful to downcast trait objects (see RpcEnvironment).
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 04/16] move src/server/environment.rs to proxmox-rest-server crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (2 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 03/16] move src/tools/daemon.rs " Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 05/16] move src/server/formatter.rs " Dietmar Maurer
` (12 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
{src/server => proxmox-rest-server/src}/environment.rs | 0
proxmox-rest-server/src/lib.rs | 3 +++
src/server/mod.rs | 3 ---
src/server/rest.rs | 3 +--
4 files changed, 4 insertions(+), 5 deletions(-)
rename {src/server => proxmox-rest-server/src}/environment.rs (100%)
diff --git a/src/server/environment.rs b/proxmox-rest-server/src/environment.rs
similarity index 100%
rename from src/server/environment.rs
rename to proxmox-rest-server/src/environment.rs
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 21a91115..42e78d89 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -6,6 +6,9 @@ use proxmox::tools::fd::Fd;
pub mod daemon;
+mod environment;
+pub use environment::*;
+
mod state;
pub use state::*;
diff --git a/src/server/mod.rs b/src/server/mod.rs
index 69cfe7b0..c426383f 100644
--- a/src/server/mod.rs
+++ b/src/server/mod.rs
@@ -46,9 +46,6 @@ pub fn our_ctrl_sock() -> String {
ctrl_sock_from_pid(*PID)
}
-mod environment;
-pub use environment::*;
-
mod upid;
pub use upid::*;
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 0c4df906..6af6fa55 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -34,9 +34,8 @@ use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
use pbs_api_types::{Authid, Userid};
-use proxmox_rest_server::{ApiConfig, FileLogger, FileLogOptions, AuthError};
+use proxmox_rest_server::{ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment};
-use super::environment::RestEnvironment;
use super::formatter::*;
use crate::auth_helpers::*;
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 05/16] move src/server/formatter.rs to proxmox-rest-server crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (3 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 04/16] move src/server/environment.rs to proxmox-rest-server crate Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 06/16] move src/tools/compression.rs " Dietmar Maurer
` (11 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
{src/server => proxmox-rest-server/src}/formatter.rs | 0
proxmox-rest-server/src/lib.rs | 1 +
src/api2/admin/datastore.rs | 3 ++-
src/api2/backup/environment.rs | 2 +-
src/api2/reader/environment.rs | 2 +-
src/server/h2service.rs | 3 ++-
src/server/mod.rs | 2 --
src/server/rest.rs | 3 +--
8 files changed, 8 insertions(+), 8 deletions(-)
rename {src/server => proxmox-rest-server/src}/formatter.rs (100%)
diff --git a/src/server/formatter.rs b/proxmox-rest-server/src/formatter.rs
similarity index 100%
rename from src/server/formatter.rs
rename to proxmox-rest-server/src/formatter.rs
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 42e78d89..bc8334ba 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -5,6 +5,7 @@ use anyhow::{bail, format_err, Error};
use proxmox::tools::fd::Fd;
pub mod daemon;
+pub mod formatter;
mod environment;
pub use environment::*;
diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index dc1c0ae9..20aca6fb 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -54,6 +54,7 @@ use pbs_tools::blocking::WrappedReaderStream;
use pbs_tools::stream::{AsyncReaderStream, AsyncChannelWriter};
use pbs_tools::json::{required_integer_param, required_string_param};
use pbs_config::CachedUserInfo;
+use proxmox_rest_server::formatter;
use crate::api2::node::rrd::create_value_from_rrd;
use crate::backup::{
@@ -1326,7 +1327,7 @@ pub fn upload_backup_log(
replace_file(&path, blob.raw_data(), CreateOptions::new())?;
// fixme: use correct formatter
- Ok(crate::server::formatter::json_response(Ok(Value::Null)))
+ Ok(formatter::json_response(Ok(Value::Null)))
}.boxed()
}
diff --git a/src/api2/backup/environment.rs b/src/api2/backup/environment.rs
index 1766639e..129ebd2b 100644
--- a/src/api2/backup/environment.rs
+++ b/src/api2/backup/environment.rs
@@ -15,10 +15,10 @@ use pbs_datastore::backup_info::{BackupDir, BackupInfo};
use pbs_datastore::dynamic_index::DynamicIndexWriter;
use pbs_datastore::fixed_index::FixedIndexWriter;
use pbs_api_types::Authid;
+use proxmox_rest_server::formatter::*;
use crate::backup::{verify_backup_dir_with_lock, DataStore};
use crate::server::WorkerTask;
-use crate::server::formatter::*;
use hyper::{Body, Response};
#[derive(Copy, Clone, Serialize)]
diff --git a/src/api2/reader/environment.rs b/src/api2/reader/environment.rs
index 64a01c4d..f7d79072 100644
--- a/src/api2/reader/environment.rs
+++ b/src/api2/reader/environment.rs
@@ -7,9 +7,9 @@ use proxmox::api::{RpcEnvironment, RpcEnvironmentType};
use pbs_datastore::backup_info::BackupDir;
use pbs_api_types::Authid;
+use proxmox_rest_server::formatter::*;
use crate::backup::DataStore;
-use crate::server::formatter::*;
use crate::server::WorkerTask;
//use proxmox::tools;
diff --git a/src/server/h2service.rs b/src/server/h2service.rs
index 332b3b1a..a79387c0 100644
--- a/src/server/h2service.rs
+++ b/src/server/h2service.rs
@@ -11,8 +11,9 @@ use hyper::{Body, Request, Response, StatusCode};
use proxmox::api::{ApiResponseFuture, HttpError, Router, RpcEnvironment};
use proxmox::http_err;
+use proxmox_rest_server::formatter::*;
+
use crate::tools;
-use crate::server::formatter::*;
use crate::server::WorkerTask;
/// Hyper Service implementation to handle stateful H2 connections.
diff --git a/src/server/mod.rs b/src/server/mod.rs
index c426383f..1fc57575 100644
--- a/src/server/mod.rs
+++ b/src/server/mod.rs
@@ -55,8 +55,6 @@ pub use worker_task::*;
mod h2service;
pub use h2service::*;
-pub mod formatter;
-
#[macro_use]
pub mod rest;
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 6af6fa55..8beecb55 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -35,8 +35,7 @@ use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
use pbs_api_types::{Authid, Userid};
use proxmox_rest_server::{ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment};
-
-use super::formatter::*;
+use proxmox_rest_server::formatter::*;
use crate::auth_helpers::*;
use pbs_config::CachedUserInfo;
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 06/16] move src/tools/compression.rs to proxmox-rest-server crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (4 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 05/16] move src/server/formatter.rs " Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 07/16] move normalize_uri_path and extract_cookie " Dietmar Maurer
` (10 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
{src/tools => proxmox-rest-server/src}/compression.rs | 0
proxmox-rest-server/src/lib.rs | 3 +++
src/server/rest.rs | 8 +++++---
src/tools/mod.rs | 1 -
4 files changed, 8 insertions(+), 4 deletions(-)
rename {src/tools => proxmox-rest-server/src}/compression.rs (100%)
diff --git a/src/tools/compression.rs b/proxmox-rest-server/src/compression.rs
similarity index 100%
rename from src/tools/compression.rs
rename to proxmox-rest-server/src/compression.rs
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index bc8334ba..069d80b4 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -4,6 +4,9 @@ use anyhow::{bail, format_err, Error};
use proxmox::tools::fd::Fd;
+mod compression;
+pub use compression::*;
+
pub mod daemon;
pub mod formatter;
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 8beecb55..7e2e645c 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -34,13 +34,15 @@ use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
use pbs_api_types::{Authid, Userid};
-use proxmox_rest_server::{ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment};
+use proxmox_rest_server::{
+ ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment, CompressionMethod,
+};
use proxmox_rest_server::formatter::*;
-use crate::auth_helpers::*;
use pbs_config::CachedUserInfo;
+
+use crate::auth_helpers::*;
use crate::tools;
-use crate::tools::compression::CompressionMethod;
extern "C" {
fn tzset();
diff --git a/src/tools/mod.rs b/src/tools/mod.rs
index 8fd441b5..f2576b08 100644
--- a/src/tools/mod.rs
+++ b/src/tools/mod.rs
@@ -14,7 +14,6 @@ use proxmox_http::{
pub mod apt;
pub mod async_io;
-pub mod compression;
pub mod config;
pub mod disks;
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 07/16] move normalize_uri_path and extract_cookie to proxmox-rest-server crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (5 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 06/16] move src/tools/compression.rs " Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 08/16] rest server: simplify get_index() method signature Dietmar Maurer
` (9 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
proxmox-rest-server/Cargo.toml | 1 +
proxmox-rest-server/src/lib.rs | 47 ++++++++++++++++++++++++++++++++++
src/server/auth.rs | 5 ++--
src/server/h2service.rs | 4 +--
src/server/rest.rs | 6 ++---
src/tools/mod.rs | 46 ---------------------------------
6 files changed, 55 insertions(+), 54 deletions(-)
diff --git a/proxmox-rest-server/Cargo.toml b/proxmox-rest-server/Cargo.toml
index 4d1f1459..c19dfb0b 100644
--- a/proxmox-rest-server/Cargo.toml
+++ b/proxmox-rest-server/Cargo.toml
@@ -15,6 +15,7 @@ lazy_static = "1.4"
libc = "0.2"
log = "0.4"
nix = "0.19.1"
+percent-encoding = "2.1"
serde = { version = "1.0", features = [] }
serde_json = "1.0"
tokio = { version = "1.6", features = ["signal", "process"] }
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 069d80b4..9107a03f 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -88,3 +88,50 @@ pub fn socketpair() -> Result<(Fd, Fd), Error> {
Ok((Fd(pa), Fd(pb)))
}
+
+/// Extract a specific cookie from cookie header.
+/// We assume cookie_name is already url encoded.
+pub fn extract_cookie(cookie: &str, cookie_name: &str) -> Option<String> {
+ for pair in cookie.split(';') {
+ let (name, value) = match pair.find('=') {
+ Some(i) => (pair[..i].trim(), pair[(i + 1)..].trim()),
+ None => return None, // Cookie format error
+ };
+
+ if name == cookie_name {
+ use percent_encoding::percent_decode;
+ if let Ok(value) = percent_decode(value.as_bytes()).decode_utf8() {
+ return Some(value.into());
+ } else {
+ return None; // Cookie format error
+ }
+ }
+ }
+
+ None
+}
+
+/// normalize uri path
+///
+/// Do not allow ".", "..", or hidden files ".XXXX"
+/// Also remove empty path components
+pub fn normalize_uri_path(path: &str) -> Result<(String, Vec<&str>), Error> {
+ let items = path.split('/');
+
+ let mut path = String::new();
+ let mut components = vec![];
+
+ for name in items {
+ if name.is_empty() {
+ continue;
+ }
+ if name.starts_with('.') {
+ bail!("Path contains illegal components.");
+ }
+ path.push('/');
+ path.push_str(name);
+ components.push(name);
+ }
+
+ Ok((path, components))
+}
diff --git a/src/server/auth.rs b/src/server/auth.rs
index 321d1e02..e4cf9034 100644
--- a/src/server/auth.rs
+++ b/src/server/auth.rs
@@ -6,10 +6,9 @@ use std::sync::Arc;
use pbs_tools::ticket::{self, Ticket};
use pbs_config::{token_shadow, CachedUserInfo};
use pbs_api_types::{Authid, Userid};
-use proxmox_rest_server::{ApiAuth, AuthError};
+use proxmox_rest_server::{ApiAuth, AuthError, extract_cookie};
use crate::auth_helpers::*;
-use crate::tools;
use hyper::header;
use percent_encoding::percent_decode_str;
@@ -33,7 +32,7 @@ impl UserApiAuth {
fn extract_auth_data(headers: &http::HeaderMap) -> Option<AuthData> {
if let Some(raw_cookie) = headers.get(header::COOKIE) {
if let Ok(cookie) = raw_cookie.to_str() {
- if let Some(ticket) = tools::extract_cookie(cookie, "PBSAuthCookie") {
+ if let Some(ticket) = extract_cookie(cookie, "PBSAuthCookie") {
let csrf_token = match headers.get("CSRFPreventionToken").map(|v| v.to_str()) {
Some(Ok(v)) => Some(v.to_owned()),
_ => None,
diff --git a/src/server/h2service.rs b/src/server/h2service.rs
index a79387c0..4e3024aa 100644
--- a/src/server/h2service.rs
+++ b/src/server/h2service.rs
@@ -11,9 +11,9 @@ use hyper::{Body, Request, Response, StatusCode};
use proxmox::api::{ApiResponseFuture, HttpError, Router, RpcEnvironment};
use proxmox::http_err;
+use proxmox_rest_server::normalize_uri_path;
use proxmox_rest_server::formatter::*;
-use crate::tools;
use crate::server::WorkerTask;
/// Hyper Service implementation to handle stateful H2 connections.
@@ -44,7 +44,7 @@ impl <E: RpcEnvironment + Clone> H2Service<E> {
let method = parts.method.clone();
- let (path, components) = match tools::normalize_uri_path(parts.uri.path()) {
+ let (path, components) = match normalize_uri_path(parts.uri.path()) {
Ok((p,c)) => (p, c),
Err(err) => return future::err(http_err!(BAD_REQUEST, "{}", err)).boxed(),
};
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 7e2e645c..c63b3df8 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -36,13 +36,13 @@ use pbs_tools::stream::AsyncReaderStream;
use pbs_api_types::{Authid, Userid};
use proxmox_rest_server::{
ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment, CompressionMethod,
+ extract_cookie, normalize_uri_path,
};
use proxmox_rest_server::formatter::*;
use pbs_config::CachedUserInfo;
use crate::auth_helpers::*;
-use crate::tools;
extern "C" {
fn tzset();
@@ -645,7 +645,7 @@ async fn handle_static_file_download(
fn extract_lang_header(headers: &http::HeaderMap) -> Option<String> {
if let Some(Ok(cookie)) = headers.get("COOKIE").map(|v| v.to_str()) {
- return tools::extract_cookie(cookie, "PBSLangCookie");
+ return extract_cookie(cookie, "PBSLangCookie");
}
None
}
@@ -669,7 +669,7 @@ async fn handle_request(
) -> Result<Response<Body>, Error> {
let (parts, body) = req.into_parts();
let method = parts.method.clone();
- let (path, components) = tools::normalize_uri_path(parts.uri.path())?;
+ let (path, components) = normalize_uri_path(parts.uri.path())?;
let comp_len = components.len();
diff --git a/src/tools/mod.rs b/src/tools/mod.rs
index f2576b08..5dc129f0 100644
--- a/src/tools/mod.rs
+++ b/src/tools/mod.rs
@@ -49,27 +49,6 @@ pub fn assert_if_modified(digest1: &str, digest2: &str) -> Result<(), Error> {
Ok(())
}
-/// Extract a specific cookie from cookie header.
-/// We assume cookie_name is already url encoded.
-pub fn extract_cookie(cookie: &str, cookie_name: &str) -> Option<String> {
- for pair in cookie.split(';') {
- let (name, value) = match pair.find('=') {
- Some(i) => (pair[..i].trim(), pair[(i + 1)..].trim()),
- None => return None, // Cookie format error
- };
-
- if name == cookie_name {
- use percent_encoding::percent_decode;
- if let Ok(value) = percent_decode(value.as_bytes()).decode_utf8() {
- return Some(value.into());
- } else {
- return None; // Cookie format error
- }
- }
- }
-
- None
-}
/// Detect modified configuration files
///
@@ -81,31 +60,6 @@ pub fn detect_modified_configuration_file(digest1: &[u8;32], digest2: &[u8;32])
Ok(())
}
-/// normalize uri path
-///
-/// Do not allow ".", "..", or hidden files ".XXXX"
-/// Also remove empty path components
-pub fn normalize_uri_path(path: &str) -> Result<(String, Vec<&str>), Error> {
- let items = path.split('/');
-
- let mut path = String::new();
- let mut components = vec![];
-
- for name in items {
- if name.is_empty() {
- continue;
- }
- if name.starts_with('.') {
- bail!("Path contains illegal components.");
- }
- path.push('/');
- path.push_str(name);
- components.push(name);
- }
-
- Ok((path, components))
-}
-
/// An easy way to convert types to Any
///
/// Mostly useful to downcast trait objects (see RpcEnvironment).
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 08/16] rest server: simplify get_index() method signature
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (6 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 07/16] move normalize_uri_path and extract_cookie " Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 09/16] make get_index and ApiConfig property (callback) Dietmar Maurer
` (8 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
src/server/rest.rs | 36 ++++++++++++++++++++----------------
1 file changed, 20 insertions(+), 16 deletions(-)
diff --git a/src/server/rest.rs b/src/server/rest.rs
index c63b3df8..3cc6bccb 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -33,7 +33,7 @@ use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
-use pbs_api_types::{Authid, Userid};
+use pbs_api_types::Authid;
use proxmox_rest_server::{
ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment, CompressionMethod,
extract_cookie, normalize_uri_path,
@@ -469,12 +469,27 @@ pub async fn handle_api_request<Env: RpcEnvironment, S: 'static + BuildHasher +
}
fn get_index(
- userid: Option<Userid>,
- csrf_token: Option<String>,
+ auth_id: Option<String>,
language: Option<String>,
api: &Arc<ApiConfig>,
parts: Parts,
) -> Response<Body> {
+
+ let (userid, csrf_token) = match auth_id {
+ Some(auth_id) => {
+ let auth_id = auth_id.parse::<Authid>();
+ match auth_id {
+ Ok(auth_id) if !auth_id.is_token() => {
+ let userid = auth_id.user().clone();
+ let new_csrf_token = assemble_csrf_prevention_token(csrf_secret(), &userid);
+ (Some(userid), Some(new_csrf_token))
+ }
+ _ => (None, None)
+ }
+ }
+ None => (None, None),
+ };
+
let nodename = proxmox::tools::nodename();
let user = userid.as_ref().map(|u| u.as_str()).unwrap_or("");
@@ -787,25 +802,14 @@ async fn handle_request(
let language = extract_lang_header(&parts.headers);
match auth.check_auth(&parts.headers, &method) {
Ok(auth_id) => {
- let auth_id: Authid = auth_id.parse()?;
- if !auth_id.is_token() {
- let userid = auth_id.user();
- let new_csrf_token = assemble_csrf_prevention_token(csrf_secret(), userid);
- return Ok(get_index(
- Some(userid.clone()),
- Some(new_csrf_token),
- language,
- &api,
- parts,
- ));
- }
+ return Ok(get_index(Some(auth_id), language, &api, parts));
}
Err(AuthError::Generic(_)) => {
tokio::time::sleep_until(Instant::from_std(delay_unauth_time)).await;
}
Err(AuthError::NoData) => {}
}
- return Ok(get_index(None, None, language, &api, parts));
+ return Ok(get_index(None, language, &api, parts));
} else {
let filename = api.find_alias(&components);
let compression = extract_compression_method(&parts.headers);
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 09/16] make get_index and ApiConfig property (callback)
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (7 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 08/16] rest server: simplify get_index() method signature Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 10/16] rest server: return UserInformation from ApiAuth::check_auth Dietmar Maurer
` (7 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
proxmox-rest-server/src/api_config.rs | 18 +++++-
src/bin/proxmox-backup-api.rs | 21 +++++++
src/bin/proxmox-backup-proxy.rs | 81 ++++++++++++++++++++++++++-
src/bin/proxmox-restore-daemon.rs | 22 +++++++-
src/server/rest.rs | 80 +-------------------------
5 files changed, 142 insertions(+), 80 deletions(-)
diff --git a/proxmox-rest-server/src/api_config.rs b/proxmox-rest-server/src/api_config.rs
index a319e204..fee94e88 100644
--- a/proxmox-rest-server/src/api_config.rs
+++ b/proxmox-rest-server/src/api_config.rs
@@ -5,7 +5,9 @@ use std::fs::metadata;
use std::sync::{Arc, Mutex, RwLock};
use anyhow::{bail, Error, format_err};
-use hyper::Method;
+use hyper::{Method, Body, Response};
+use hyper::http::request::Parts;
+
use handlebars::Handlebars;
use serde::Serialize;
@@ -14,6 +16,8 @@ use proxmox::tools::fs::{create_path, CreateOptions};
use crate::{ApiAuth, FileLogger, FileLogOptions, CommandoSocket};
+pub type GetIndexFn = fn(Option<String>, Option<String>, &ApiConfig, Parts) -> Response<Body>;
+
pub struct ApiConfig {
basedir: PathBuf,
router: &'static Router,
@@ -23,6 +27,7 @@ pub struct ApiConfig {
template_files: RwLock<HashMap<String, (SystemTime, PathBuf)>>,
request_log: Option<Arc<Mutex<FileLogger>>>,
pub api_auth: Arc<dyn ApiAuth + Send + Sync>,
+ get_index_fn: GetIndexFn,
}
impl ApiConfig {
@@ -31,6 +36,7 @@ impl ApiConfig {
router: &'static Router,
env_type: RpcEnvironmentType,
api_auth: Arc<dyn ApiAuth + Send + Sync>,
+ get_index_fn: GetIndexFn,
) -> Result<Self, Error> {
Ok(Self {
basedir: basedir.into(),
@@ -41,9 +47,19 @@ impl ApiConfig {
template_files: RwLock::new(HashMap::new()),
request_log: None,
api_auth,
+ get_index_fn,
})
}
+ pub fn get_index(
+ &self,
+ auth_id: Option<String>,
+ language: Option<String>,
+ parts: Parts,
+ ) -> Response<Body> {
+ (self.get_index_fn)(auth_id, language, self, parts)
+ }
+
pub fn find_method(
&self,
components: &[&str],
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index 17b6f184..3bc02639 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -1,5 +1,9 @@
use anyhow::{bail, Error};
use futures::*;
+use http::request::Parts;
+use http::Response;
+use hyper::{Body, StatusCode};
+use hyper::header;
use proxmox::try_block;
use proxmox::api::RpcEnvironmentType;
@@ -27,6 +31,22 @@ fn main() {
}
}
+fn get_index(
+ _auth_id: Option<String>,
+ _language: Option<String>,
+ _api: &ApiConfig,
+ _parts: Parts,
+) -> Response<Body> {
+
+ let index = "<center><h1>Proxmox Backup API Server</h1></center>";
+
+ Response::builder()
+ .status(StatusCode::OK)
+ .header(header::CONTENT_TYPE, "text/html")
+ .body(index.into())
+ .unwrap()
+}
+
async fn run() -> Result<(), Error> {
if let Err(err) = syslog::init(
syslog::Facility::LOG_DAEMON,
@@ -65,6 +85,7 @@ async fn run() -> Result<(), Error> {
&proxmox_backup::api2::ROUTER,
RpcEnvironmentType::PRIVILEGED,
default_api_auth(),
+ get_index,
)?;
let backup_user = pbs_config::backup_user()?;
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index d4ac2a85..6a4fddef 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -4,10 +4,15 @@ use std::os::unix::io::AsRawFd;
use anyhow::{bail, format_err, Error};
use futures::*;
+use http::request::Parts;
+use http::Response;
+use hyper::{Body, StatusCode};
+use hyper::header;
+use url::form_urlencoded;
use openssl::ssl::{SslMethod, SslAcceptor, SslFiletype};
use tokio_stream::wrappers::ReceiverStream;
-use serde_json::Value;
+use serde_json::{json, Value};
use proxmox::try_block;
use proxmox::api::RpcEnvironmentType;
@@ -73,6 +78,79 @@ fn main() -> Result<(), Error> {
pbs_runtime::main(run())
}
+fn get_index(
+ auth_id: Option<String>,
+ language: Option<String>,
+ api: &ApiConfig,
+ parts: Parts,
+) -> Response<Body> {
+
+ let (userid, csrf_token) = match auth_id {
+ Some(auth_id) => {
+ let auth_id = auth_id.parse::<Authid>();
+ match auth_id {
+ Ok(auth_id) if !auth_id.is_token() => {
+ let userid = auth_id.user().clone();
+ let new_csrf_token = assemble_csrf_prevention_token(csrf_secret(), &userid);
+ (Some(userid), Some(new_csrf_token))
+ }
+ _ => (None, None)
+ }
+ }
+ None => (None, None),
+ };
+
+ let nodename = proxmox::tools::nodename();
+ let user = userid.as_ref().map(|u| u.as_str()).unwrap_or("");
+
+ let csrf_token = csrf_token.unwrap_or_else(|| String::from(""));
+
+ let mut debug = false;
+ let mut template_file = "index";
+
+ if let Some(query_str) = parts.uri.query() {
+ for (k, v) in form_urlencoded::parse(query_str.as_bytes()).into_owned() {
+ if k == "debug" && v != "0" && v != "false" {
+ debug = true;
+ } else if k == "console" {
+ template_file = "console";
+ }
+ }
+ }
+
+ let mut lang = String::from("");
+ if let Some(language) = language {
+ if Path::new(&format!("/usr/share/pbs-i18n/pbs-lang-{}.js", language)).exists() {
+ lang = language;
+ }
+ }
+
+ let data = json!({
+ "NodeName": nodename,
+ "UserName": user,
+ "CSRFPreventionToken": csrf_token,
+ "language": lang,
+ "debug": debug,
+ });
+
+ let (ct, index) = match api.render_template(template_file, &data) {
+ Ok(index) => ("text/html", index),
+ Err(err) => ("text/plain", format!("Error rendering template: {}", err)),
+ };
+
+ let mut resp = Response::builder()
+ .status(StatusCode::OK)
+ .header(header::CONTENT_TYPE, ct)
+ .body(index.into())
+ .unwrap();
+
+ if let Some(userid) = userid {
+ resp.extensions_mut().insert(Authid::from((userid, None)));
+ }
+
+ resp
+}
+
async fn run() -> Result<(), Error> {
if let Err(err) = syslog::init(
syslog::Facility::LOG_DAEMON,
@@ -93,6 +171,7 @@ async fn run() -> Result<(), Error> {
&proxmox_backup::api2::ROUTER,
RpcEnvironmentType::PUBLIC,
default_api_auth(),
+ get_index,
)?;
config.add_alias("novnc", "/usr/share/novnc-pve");
diff --git a/src/bin/proxmox-restore-daemon.rs b/src/bin/proxmox-restore-daemon.rs
index c5122cff..d9a8eff0 100644
--- a/src/bin/proxmox-restore-daemon.rs
+++ b/src/bin/proxmox-restore-daemon.rs
@@ -13,6 +13,10 @@ use lazy_static::lazy_static;
use log::{error, info};
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
+use http::request::Parts;
+use http::Response;
+use hyper::{Body, StatusCode};
+use hyper::header;
use proxmox::api::RpcEnvironmentType;
@@ -89,13 +93,29 @@ fn setup_system_env() -> Result<(), Error> {
Ok(())
}
+fn get_index(
+ _auth_id: Option<String>,
+ _language: Option<String>,
+ _api: &ApiConfig,
+ _parts: Parts,
+) -> Response<Body> {
+
+ let index = "<center><h1>Proxmox Backup Restore Daemon/h1></center>";
+
+ Response::builder()
+ .status(StatusCode::OK)
+ .header(header::CONTENT_TYPE, "text/html")
+ .body(index.into())
+ .unwrap()
+}
+
async fn run() -> Result<(), Error> {
watchdog_init();
let auth_config = Arc::new(
auth::ticket_auth().map_err(|err| format_err!("reading ticket file failed: {}", err))?,
);
- let config = ApiConfig::new("", &ROUTER, RpcEnvironmentType::PUBLIC, auth_config)?;
+ let config = ApiConfig::new("", &ROUTER, RpcEnvironmentType::PUBLIC, auth_config, get_index)?;
let rest_server = RestServer::new(config);
let vsock_fd = get_vsock_fd()?;
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 3cc6bccb..9ed0eb32 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -15,7 +15,7 @@ use hyper::http::request::Parts;
use hyper::{Body, Request, Response, StatusCode};
use lazy_static::lazy_static;
use regex::Regex;
-use serde_json::{json, Value};
+use serde_json::Value;
use tokio::fs::File;
use tokio::time::Instant;
use url::form_urlencoded;
@@ -42,8 +42,6 @@ use proxmox_rest_server::formatter::*;
use pbs_config::CachedUserInfo;
-use crate::auth_helpers::*;
-
extern "C" {
fn tzset();
}
@@ -468,78 +466,6 @@ pub async fn handle_api_request<Env: RpcEnvironment, S: 'static + BuildHasher +
Ok(resp)
}
-fn get_index(
- auth_id: Option<String>,
- language: Option<String>,
- api: &Arc<ApiConfig>,
- parts: Parts,
-) -> Response<Body> {
-
- let (userid, csrf_token) = match auth_id {
- Some(auth_id) => {
- let auth_id = auth_id.parse::<Authid>();
- match auth_id {
- Ok(auth_id) if !auth_id.is_token() => {
- let userid = auth_id.user().clone();
- let new_csrf_token = assemble_csrf_prevention_token(csrf_secret(), &userid);
- (Some(userid), Some(new_csrf_token))
- }
- _ => (None, None)
- }
- }
- None => (None, None),
- };
-
- let nodename = proxmox::tools::nodename();
- let user = userid.as_ref().map(|u| u.as_str()).unwrap_or("");
-
- let csrf_token = csrf_token.unwrap_or_else(|| String::from(""));
-
- let mut debug = false;
- let mut template_file = "index";
-
- if let Some(query_str) = parts.uri.query() {
- for (k, v) in form_urlencoded::parse(query_str.as_bytes()).into_owned() {
- if k == "debug" && v != "0" && v != "false" {
- debug = true;
- } else if k == "console" {
- template_file = "console";
- }
- }
- }
-
- let mut lang = String::from("");
- if let Some(language) = language {
- if Path::new(&format!("/usr/share/pbs-i18n/pbs-lang-{}.js", language)).exists() {
- lang = language;
- }
- }
-
- let data = json!({
- "NodeName": nodename,
- "UserName": user,
- "CSRFPreventionToken": csrf_token,
- "language": lang,
- "debug": debug,
- });
-
- let (ct, index) = match api.render_template(template_file, &data) {
- Ok(index) => ("text/html", index),
- Err(err) => ("text/plain", format!("Error rendering template: {}", err)),
- };
-
- let mut resp = Response::builder()
- .status(StatusCode::OK)
- .header(header::CONTENT_TYPE, ct)
- .body(index.into())
- .unwrap();
-
- if let Some(userid) = userid {
- resp.extensions_mut().insert(Authid::from((userid, None)));
- }
-
- resp
-}
fn extension_to_content_type(filename: &Path) -> (&'static str, bool) {
if let Some(ext) = filename.extension().and_then(|osstr| osstr.to_str()) {
@@ -802,14 +728,14 @@ async fn handle_request(
let language = extract_lang_header(&parts.headers);
match auth.check_auth(&parts.headers, &method) {
Ok(auth_id) => {
- return Ok(get_index(Some(auth_id), language, &api, parts));
+ return Ok(api.get_index(Some(auth_id), language, parts));
}
Err(AuthError::Generic(_)) => {
tokio::time::sleep_until(Instant::from_std(delay_unauth_time)).await;
}
Err(AuthError::NoData) => {}
}
- return Ok(get_index(None, language, &api, parts));
+ return Ok(api.get_index(None, language, parts));
} else {
let filename = api.find_alias(&components);
let compression = extract_compression_method(&parts.headers);
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 10/16] rest server: return UserInformation from ApiAuth::check_auth
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (8 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 09/16] make get_index and ApiConfig property (callback) Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 11/16] rest server: do not use pbs_api_types::Authid Dietmar Maurer
` (6 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
This need impl UserInformation for Arc<CachedUserInfo> which is implemented
with proxmox 0.13.2
---
proxmox-rest-server/src/lib.rs | 3 ++-
src/bin/proxmox_restore_daemon/auth.rs | 16 ++++++++++++++--
src/server/auth.rs | 9 ++++++---
src/server/rest.rs | 23 +++++++++++++++++------
4 files changed, 39 insertions(+), 12 deletions(-)
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 9107a03f..55a10ca6 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -3,6 +3,7 @@ use std::os::unix::io::RawFd;
use anyhow::{bail, format_err, Error};
use proxmox::tools::fd::Fd;
+use proxmox::api::UserInformation;
mod compression;
pub use compression::*;
@@ -41,7 +42,7 @@ pub trait ApiAuth {
&self,
headers: &http::HeaderMap,
method: &hyper::Method,
- ) -> Result<String, AuthError>;
+ ) -> Result<(String, Box<dyn UserInformation + Sync + Send>), AuthError>;
}
static mut SHUTDOWN_REQUESTED: bool = false;
diff --git a/src/bin/proxmox_restore_daemon/auth.rs b/src/bin/proxmox_restore_daemon/auth.rs
index ea1dabe6..6d6e9c58 100644
--- a/src/bin/proxmox_restore_daemon/auth.rs
+++ b/src/bin/proxmox_restore_daemon/auth.rs
@@ -4,10 +4,22 @@ use std::io::prelude::*;
use anyhow::{bail, format_err, Error};
+use proxmox::api::UserInformation;
+
use proxmox_rest_server::{ApiAuth, AuthError};
const TICKET_FILE: &str = "/ticket";
+struct SimpleUserInformation {}
+
+impl UserInformation for SimpleUserInformation {
+ fn is_superuser(&self, userid: &str) -> bool {
+ userid == "root@pam"
+ }
+ fn is_group_member(&self, _userid: &str, _group: &str) -> bool { false }
+ fn lookup_privs(&self, _userid: &str, _path: &[&str]) -> u64 { 0 }
+}
+
pub struct StaticAuth {
ticket: String,
}
@@ -17,10 +29,10 @@ impl ApiAuth for StaticAuth {
&self,
headers: &http::HeaderMap,
_method: &hyper::Method,
- ) -> Result<String, AuthError> {
+ ) -> Result<(String, Box<dyn UserInformation + Send + Sync>), AuthError> {
match headers.get(hyper::header::AUTHORIZATION) {
Some(header) if header.to_str().unwrap_or("") == &self.ticket => {
- Ok(String::from("root@pam"))
+ Ok((String::from("root@pam"), Box::new(SimpleUserInformation {})))
}
_ => {
return Err(AuthError::Generic(format_err!(
diff --git a/src/server/auth.rs b/src/server/auth.rs
index e4cf9034..90252435 100644
--- a/src/server/auth.rs
+++ b/src/server/auth.rs
@@ -3,6 +3,8 @@ use anyhow::format_err;
use std::sync::Arc;
+use proxmox::api::UserInformation;
+
use pbs_tools::ticket::{self, Ticket};
use pbs_config::{token_shadow, CachedUserInfo};
use pbs_api_types::{Authid, Userid};
@@ -56,11 +58,12 @@ impl UserApiAuth {
}
impl ApiAuth for UserApiAuth {
+
fn check_auth(
&self,
headers: &http::HeaderMap,
method: &hyper::Method,
- ) -> Result<String, AuthError> {
+ ) -> Result<(String, Box<dyn UserInformation + Sync + Send>), AuthError> {
let user_info = CachedUserInfo::new()?;
@@ -93,7 +96,7 @@ impl ApiAuth for UserApiAuth {
}
}
- Ok(auth_id.to_string())
+ Ok((auth_id.to_string(), Box::new(user_info)))
}
Some(AuthData::ApiToken(api_token)) => {
let mut parts = api_token.splitn(2, ':');
@@ -115,7 +118,7 @@ impl ApiAuth for UserApiAuth {
token_shadow::verify_secret(&tokenid, &tokensecret)?;
- Ok(tokenid.to_string())
+ Ok((tokenid.to_string(), Box::new(user_info)))
}
None => Err(AuthError::NoData),
}
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 9ed0eb32..d87985c1 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -26,7 +26,7 @@ use proxmox::api::schema::{
};
use proxmox::api::{
check_api_permission, ApiHandler, ApiMethod, HttpError, Permission, RpcEnvironment,
- RpcEnvironmentType,
+ RpcEnvironmentType, UserInformation,
};
use proxmox::http_err;
use proxmox::tools::fs::CreateOptions;
@@ -40,12 +40,18 @@ use proxmox_rest_server::{
};
use proxmox_rest_server::formatter::*;
-use pbs_config::CachedUserInfo;
-
extern "C" {
fn tzset();
}
+struct EmptyUserInformation {}
+
+impl UserInformation for EmptyUserInformation {
+ fn is_superuser(&self, _userid: &str) -> bool { false }
+ fn is_group_member(&self, _userid: &str, _group: &str) -> bool { false }
+ fn lookup_privs(&self, _userid: &str, _path: &[&str]) -> u64 { 0 }
+}
+
pub struct RestServer {
pub api_config: Arc<ApiConfig>,
}
@@ -652,9 +658,14 @@ async fn handle_request(
}
}
+ let mut user_info: Box<dyn UserInformation + Send + Sync> = Box::new(EmptyUserInformation {});
+
if auth_required {
match auth.check_auth(&parts.headers, &method) {
- Ok(authid) => rpcenv.set_auth_id(Some(authid)),
+ Ok((authid, info)) => {
+ rpcenv.set_auth_id(Some(authid));
+ user_info = info;
+ }
Err(auth_err) => {
let err = match auth_err {
AuthError::Generic(err) => err,
@@ -683,7 +694,7 @@ async fn handle_request(
}
Some(api_method) => {
let auth_id = rpcenv.get_auth_id();
- let user_info = CachedUserInfo::new()?;
+ let user_info = user_info;
if !check_api_permission(
api_method.access.permission,
@@ -727,7 +738,7 @@ async fn handle_request(
if comp_len == 0 {
let language = extract_lang_header(&parts.headers);
match auth.check_auth(&parts.headers, &method) {
- Ok(auth_id) => {
+ Ok((auth_id, _user_info)) => {
return Ok(api.get_index(Some(auth_id), language, parts));
}
Err(AuthError::Generic(_)) => {
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 11/16] rest server: do not use pbs_api_types::Authid
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (9 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 10/16] rest server: return UserInformation from ApiAuth::check_auth Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 12/16] rest server: cleanup auth-log handling Dietmar Maurer
` (5 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
src/server/rest.rs | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/src/server/rest.rs b/src/server/rest.rs
index d87985c1..2083743b 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -33,7 +33,6 @@ use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
-use pbs_api_types::Authid;
use proxmox_rest_server::{
ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment, CompressionMethod,
extract_cookie, normalize_uri_path,
@@ -44,6 +43,8 @@ extern "C" {
fn tzset();
}
+struct AuthStringExtension(String);
+
struct EmptyUserInformation {}
impl UserInformation for EmptyUserInformation {
@@ -176,8 +177,8 @@ fn log_response(
);
}
if let Some(logfile) = logfile {
- let auth_id = match resp.extensions().get::<Authid>() {
- Some(auth_id) => auth_id.to_string(),
+ let auth_id = match resp.extensions().get::<AuthStringExtension>() {
+ Some(AuthStringExtension(auth_id)) => auth_id.clone(),
None => "-".to_string(),
};
let now = proxmox::tools::time::epoch_i64();
@@ -198,6 +199,7 @@ fn log_response(
));
}
}
+
pub fn auth_logger() -> Result<FileLogger, Error> {
let backup_user = pbs_config::backup_user()?;
@@ -720,8 +722,7 @@ async fn handle_request(
};
if let Some(auth_id) = auth_id {
- let auth_id: Authid = auth_id.parse()?;
- response.extensions_mut().insert(auth_id);
+ response.extensions_mut().insert(AuthStringExtension(auth_id));
}
return Ok(response);
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 12/16] rest server: cleanup auth-log handling
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (10 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 11/16] rest server: do not use pbs_api_types::Authid Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 13/16] move src/server/rest.rs to proxmox-rest-server crate Dietmar Maurer
` (4 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
Handle auth logs the same way as access log.
- Configure with ApiConfig
- CommandoSocket command to reload auth-logs "api-auth-log-reopen"
Inside API calls, we now access the ApiConfig using the RestEnvironment.
The openid_login api now also logs failed logins and return http_err!(UNAUTHORIZED, ..)
on failed logins.
---
proxmox-rest-server/src/api_config.rs | 45 ++++++++-
proxmox-rest-server/src/environment.rs | 49 ++++++++-
src/api2/access/mod.rs | 26 ++---
src/api2/access/openid.rs | 134 ++++++++++++++-----------
src/bin/proxmox-backup-api.rs | 8 ++
src/bin/proxmox-backup-proxy.rs | 31 +++++-
src/server/rest.rs | 30 +-----
7 files changed, 215 insertions(+), 108 deletions(-)
diff --git a/proxmox-rest-server/src/api_config.rs b/proxmox-rest-server/src/api_config.rs
index fee94e88..e09f7af9 100644
--- a/proxmox-rest-server/src/api_config.rs
+++ b/proxmox-rest-server/src/api_config.rs
@@ -26,6 +26,7 @@ pub struct ApiConfig {
templates: RwLock<Handlebars<'static>>,
template_files: RwLock<HashMap<String, (SystemTime, PathBuf)>>,
request_log: Option<Arc<Mutex<FileLogger>>>,
+ auth_log: Option<Arc<Mutex<FileLogger>>>,
pub api_auth: Arc<dyn ApiAuth + Send + Sync>,
get_index_fn: GetIndexFn,
}
@@ -46,6 +47,7 @@ impl ApiConfig {
templates: RwLock::new(Handlebars::new()),
template_files: RwLock::new(HashMap::new()),
request_log: None,
+ auth_log: None,
api_auth,
get_index_fn,
})
@@ -172,7 +174,7 @@ impl ApiConfig {
self.request_log = Some(Arc::clone(&request_log));
commando_sock.register_command("api-access-log-reopen".into(), move |_args| {
- println!("re-opening log file");
+ println!("re-opening access-log file");
request_log.lock().unwrap().reopen()?;
Ok(serde_json::Value::Null)
})?;
@@ -180,7 +182,46 @@ impl ApiConfig {
Ok(())
}
- pub fn get_file_log(&self) -> Option<&Arc<Mutex<FileLogger>>> {
+ pub fn enable_auth_log<P>(
+ &mut self,
+ path: P,
+ dir_opts: Option<CreateOptions>,
+ file_opts: Option<CreateOptions>,
+ commando_sock: &mut CommandoSocket,
+ ) -> Result<(), Error>
+ where
+ P: Into<PathBuf>
+ {
+ let path: PathBuf = path.into();
+ if let Some(base) = path.parent() {
+ if !base.exists() {
+ create_path(base, None, dir_opts).map_err(|err| format_err!("{}", err))?;
+ }
+ }
+
+ let logger_options = FileLogOptions {
+ append: true,
+ prefix_time: true,
+ file_opts: file_opts.unwrap_or(CreateOptions::default()),
+ ..Default::default()
+ };
+ let auth_log = Arc::new(Mutex::new(FileLogger::new(&path, logger_options)?));
+ self.auth_log = Some(Arc::clone(&auth_log));
+
+ commando_sock.register_command("api-auth-log-reopen".into(), move |_args| {
+ println!("re-opening auth-log file");
+ auth_log.lock().unwrap().reopen()?;
+ Ok(serde_json::Value::Null)
+ })?;
+
+ Ok(())
+ }
+
+ pub fn get_access_log(&self) -> Option<&Arc<Mutex<FileLogger>>> {
self.request_log.as_ref()
}
+
+ pub fn get_auth_log(&self) -> Option<&Arc<Mutex<FileLogger>>> {
+ self.auth_log.as_ref()
+ }
}
diff --git a/proxmox-rest-server/src/environment.rs b/proxmox-rest-server/src/environment.rs
index 0548b5bc..57d0d7d5 100644
--- a/proxmox-rest-server/src/environment.rs
+++ b/proxmox-rest-server/src/environment.rs
@@ -1,24 +1,65 @@
+use std::sync::Arc;
+use std::net::SocketAddr;
+
use serde_json::{json, Value};
use proxmox::api::{RpcEnvironment, RpcEnvironmentType};
+use crate::ApiConfig;
+
/// Encapsulates information about the runtime environment
pub struct RestEnvironment {
env_type: RpcEnvironmentType,
result_attributes: Value,
auth_id: Option<String>,
- client_ip: Option<std::net::SocketAddr>,
+ client_ip: Option<SocketAddr>,
+ api: Arc<ApiConfig>,
}
impl RestEnvironment {
- pub fn new(env_type: RpcEnvironmentType) -> Self {
+ pub fn new(env_type: RpcEnvironmentType, api: Arc<ApiConfig>) -> Self {
Self {
result_attributes: json!({}),
auth_id: None,
client_ip: None,
env_type,
+ api,
}
}
+
+ pub fn api_config(&self) -> &ApiConfig {
+ &self.api
+ }
+
+ pub fn log_auth(&self, auth_id: &str) {
+ let msg = format!("successful auth for user '{}'", auth_id);
+ log::info!("{}", msg);
+ if let Some(auth_logger) = self.api.get_auth_log() {
+ auth_logger.lock().unwrap().log(&msg);
+ }
+ }
+
+ pub fn log_failed_auth(&self, failed_auth_id: Option<String>, msg: &str) {
+ let msg = match (self.client_ip, failed_auth_id) {
+ (Some(peer), Some(user)) => {
+ format!("authentication failure; rhost={} user={} msg={}", peer, user, msg)
+ }
+ (Some(peer), None) => {
+ format!("authentication failure; rhost={} msg={}", peer, msg)
+ }
+ (None, Some(user)) => {
+ format!("authentication failure; rhost=unknown user={} msg={}", user, msg)
+ }
+ (None, None) => {
+ format!("authentication failure; rhost=unknown msg={}", msg)
+ }
+ };
+ log::error!("{}", msg);
+ if let Some(auth_logger) = self.api.get_auth_log() {
+ auth_logger.lock().unwrap().log(&msg);
+ }
+ }
+
}
impl RpcEnvironment for RestEnvironment {
@@ -43,11 +84,11 @@ impl RpcEnvironment for RestEnvironment {
self.auth_id.clone()
}
- fn set_client_ip(&mut self, client_ip: Option<std::net::SocketAddr>) {
+ fn set_client_ip(&mut self, client_ip: Option<SocketAddr>) {
self.client_ip = client_ip;
}
- fn get_client_ip(&self) -> Option<std::net::SocketAddr> {
+ fn get_client_ip(&self) -> Option<SocketAddr> {
self.client_ip
}
}
diff --git a/src/api2/access/mod.rs b/src/api2/access/mod.rs
index 58ac8ca4..48a90d1b 100644
--- a/src/api2/access/mod.rs
+++ b/src/api2/access/mod.rs
@@ -12,7 +12,7 @@ use proxmox::{http_err, list_subdirs_api_method};
use proxmox::{identity, sortable};
use pbs_api_types::{
- Userid, Authid, PASSWORD_SCHEMA, ACL_PATH_SCHEMA,
+ Userid, Authid, PASSWORD_SCHEMA, ACL_PATH_SCHEMA,
PRIVILEGES, PRIV_PERMISSIONS_MODIFY, PRIV_SYS_AUDIT,
};
use pbs_tools::auth::private_auth_key;
@@ -196,6 +196,12 @@ pub fn create_ticket(
tfa_challenge: Option<String>,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<Value, Error> {
+
+ use proxmox_rest_server::RestEnvironment;
+
+ let env: &RestEnvironment = rpcenv.as_any().downcast_ref::<RestEnvironment>()
+ .ok_or_else(|| format_err!("detected worng RpcEnvironment type"))?;
+
match authenticate_user(&username, &password, path, privs, port, tfa_challenge) {
Ok(AuthResult::Success) => Ok(json!({ "username": username })),
Ok(AuthResult::CreateTicket) => {
@@ -203,8 +209,7 @@ pub fn create_ticket(
let ticket = Ticket::new("PBS", &api_ticket)?.sign(private_auth_key(), None)?;
let token = assemble_csrf_prevention_token(csrf_secret(), &username);
- crate::server::rest::auth_logger()?
- .log(format!("successful auth for user '{}'", username));
+ env.log_auth(username.as_str());
Ok(json!({
"username": username,
@@ -223,20 +228,7 @@ pub fn create_ticket(
}))
}
Err(err) => {
- let client_ip = match rpcenv.get_client_ip().map(|addr| addr.ip()) {
- Some(ip) => format!("{}", ip),
- None => "unknown".into(),
- };
-
- let msg = format!(
- "authentication failure; rhost={} user={} msg={}",
- client_ip,
- username,
- err.to_string()
- );
- crate::server::rest::auth_logger()?.log(&msg);
- log::error!("{}", msg);
-
+ env.log_failed_auth(Some(username.to_string()), &err.to_string());
Err(http_err!(UNAUTHORIZED, "permission check failed."))
}
}
diff --git a/src/api2/access/openid.rs b/src/api2/access/openid.rs
index 38fab409..8fe62ad2 100644
--- a/src/api2/access/openid.rs
+++ b/src/api2/access/openid.rs
@@ -1,14 +1,13 @@
//! OpenID redirect/login API
use std::convert::TryFrom;
-use anyhow::{bail, Error};
+use anyhow::{bail, format_err, Error};
use serde_json::{json, Value};
use proxmox::api::router::{Router, SubdirMap};
use proxmox::api::{api, Permission, RpcEnvironment};
-use proxmox::{list_subdirs_api_method};
-use proxmox::{identity, sortable};
+use proxmox::{http_err, list_subdirs_api_method, identity, sortable};
use proxmox_openid::{OpenIdAuthenticator, OpenIdConfig};
@@ -80,76 +79,95 @@ pub fn openid_login(
state: String,
code: String,
redirect_url: String,
- _rpcenv: &mut dyn RpcEnvironment,
+ rpcenv: &mut dyn RpcEnvironment,
) -> Result<Value, Error> {
+ use proxmox_rest_server::RestEnvironment;
+
+ let env: &RestEnvironment = rpcenv.as_any().downcast_ref::<RestEnvironment>()
+ .ok_or_else(|| format_err!("detected worng RpcEnvironment type"))?;
+
let user_info = CachedUserInfo::new()?;
- let (realm, private_auth_state) =
- OpenIdAuthenticator::verify_public_auth_state(PROXMOX_BACKUP_RUN_DIR_M!(), &state)?;
+ let mut tested_username = None;
- let (domains, _digest) = pbs_config::domains::config()?;
- let config: OpenIdRealmConfig = domains.lookup("openid", &realm)?;
+ let result = proxmox::try_block!({
- let open_id = openid_authenticator(&config, &redirect_url)?;
+ let (realm, private_auth_state) =
+ OpenIdAuthenticator::verify_public_auth_state(PROXMOX_BACKUP_RUN_DIR_M!(), &state)?;
+
+ let (domains, _digest) = pbs_config::domains::config()?;
+ let config: OpenIdRealmConfig = domains.lookup("openid", &realm)?;
+
+ let open_id = openid_authenticator(&config, &redirect_url)?;
- let info = open_id.verify_authorization_code(&code, &private_auth_state)?;
+ let info = open_id.verify_authorization_code(&code, &private_auth_state)?;
- // eprintln!("VERIFIED {} {:?} {:?}", info.subject().as_str(), info.name(), info.email());
+ // eprintln!("VERIFIED {} {:?} {:?}", info.subject().as_str(), info.name(), info.email());
- let unique_name = match config.username_claim {
- None | Some(OpenIdUserAttribute::Subject) => info.subject().as_str(),
- Some(OpenIdUserAttribute::Username) => {
- match info.preferred_username() {
- Some(name) => name.as_str(),
- None => bail!("missing claim 'preferred_name'"),
+ let unique_name = match config.username_claim {
+ None | Some(OpenIdUserAttribute::Subject) => info.subject().as_str(),
+ Some(OpenIdUserAttribute::Username) => {
+ match info.preferred_username() {
+ Some(name) => name.as_str(),
+ None => bail!("missing claim 'preferred_name'"),
+ }
}
- }
- Some(OpenIdUserAttribute::Email) => {
- match info.email() {
- Some(name) => name.as_str(),
- None => bail!("missing claim 'email'"),
+ Some(OpenIdUserAttribute::Email) => {
+ match info.email() {
+ Some(name) => name.as_str(),
+ None => bail!("missing claim 'email'"),
+ }
}
- }
- };
-
- let user_id = Userid::try_from(format!("{}@{}", unique_name, realm))?;
-
- if !user_info.is_active_user_id(&user_id) {
- if config.autocreate.unwrap_or(false) {
- use pbs_config::user;
- let _lock = open_backup_lockfile(user::USER_CFG_LOCKFILE, None, true)?;
- let user = User {
- userid: user_id.clone(),
- comment: None,
- enable: None,
- expire: None,
- firstname: info.given_name().and_then(|n| n.get(None)).map(|n| n.to_string()),
- lastname: info.family_name().and_then(|n| n.get(None)).map(|n| n.to_string()),
- email: info.email().map(|e| e.to_string()),
- };
- let (mut config, _digest) = user::config()?;
- if config.sections.get(user.userid.as_str()).is_some() {
- bail!("autocreate user failed - '{}' already exists.", user.userid);
+ };
+
+
+ let user_id = Userid::try_from(format!("{}@{}", unique_name, realm))?;
+ tested_username = Some(unique_name.to_string());
+
+ if !user_info.is_active_user_id(&user_id) {
+ if config.autocreate.unwrap_or(false) {
+ use pbs_config::user;
+ let _lock = open_backup_lockfile(user::USER_CFG_LOCKFILE, None, true)?;
+ let user = User {
+ userid: user_id.clone(),
+ comment: None,
+ enable: None,
+ expire: None,
+ firstname: info.given_name().and_then(|n| n.get(None)).map(|n| n.to_string()),
+ lastname: info.family_name().and_then(|n| n.get(None)).map(|n| n.to_string()),
+ email: info.email().map(|e| e.to_string()),
+ };
+ let (mut config, _digest) = user::config()?;
+ if config.sections.get(user.userid.as_str()).is_some() {
+ bail!("autocreate user failed - '{}' already exists.", user.userid);
+ }
+ config.set_data(user.userid.as_str(), "user", &user)?;
+ user::save_config(&config)?;
+ } else {
+ bail!("user account '{}' missing, disabled or expired.", user_id);
}
- config.set_data(user.userid.as_str(), "user", &user)?;
- user::save_config(&config)?;
- } else {
- bail!("user account '{}' missing, disabled or expired.", user_id);
}
- }
- let api_ticket = ApiTicket::full(user_id.clone());
- let ticket = Ticket::new("PBS", &api_ticket)?.sign(private_auth_key(), None)?;
- let token = assemble_csrf_prevention_token(csrf_secret(), &user_id);
+ let api_ticket = ApiTicket::full(user_id.clone());
+ let ticket = Ticket::new("PBS", &api_ticket)?.sign(private_auth_key(), None)?;
+ let token = assemble_csrf_prevention_token(csrf_secret(), &user_id);
+
+ env.log_auth(user_id.as_str());
- crate::server::rest::auth_logger()?
- .log(format!("successful auth for user '{}'", user_id));
+ Ok(json!({
+ "username": user_id,
+ "ticket": ticket,
+ "CSRFPreventionToken": token,
+ }))
+ });
+
+ if let Err(ref err) = result {
+ let msg = err.to_string();
+ env.log_failed_auth(tested_username, &msg);
+ return Err(http_err!(UNAUTHORIZED, "{}", msg))
+ }
- Ok(json!({
- "username": user_id,
- "ticket": ticket,
- "CSRFPreventionToken": token,
- }))
+ result
}
#[api(
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index 3bc02639..7653f412 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -96,11 +96,19 @@ async fn run() -> Result<(), Error> {
config.enable_file_log(
pbs_buildcfg::API_ACCESS_LOG_FN,
+ Some(dir_opts.clone()),
+ Some(file_opts.clone()),
+ &mut commando_sock,
+ )?;
+
+ config.enable_auth_log(
+ pbs_buildcfg::API_AUTH_LOG_FN,
Some(dir_opts),
Some(file_opts),
&mut commando_sock,
)?;
+
let rest_server = RestServer::new(config);
// http server future:
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 6a4fddef..5679de43 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -196,6 +196,13 @@ async fn run() -> Result<(), Error> {
config.enable_file_log(
pbs_buildcfg::API_ACCESS_LOG_FN,
+ Some(dir_opts.clone()),
+ Some(file_opts.clone()),
+ &mut commando_sock,
+ )?;
+
+ config.enable_auth_log(
+ pbs_buildcfg::API_AUTH_LOG_FN,
Some(dir_opts),
Some(file_opts),
&mut commando_sock,
@@ -762,7 +769,7 @@ async fn schedule_task_log_rotate() {
if logrotate.rotate(max_size, None, Some(max_files))? {
println!("rotated access log, telling daemons to re-open log file");
- pbs_runtime::block_on(command_reopen_logfiles())?;
+ pbs_runtime::block_on(command_reopen_access_logfiles())?;
worker.log("API access log was rotated".to_string());
} else {
worker.log("API access log was not rotated".to_string());
@@ -772,6 +779,8 @@ async fn schedule_task_log_rotate() {
.ok_or_else(|| format_err!("could not get API auth log file names"))?;
if logrotate.rotate(max_size, None, Some(max_files))? {
+ println!("rotated auth log, telling daemons to re-open log file");
+ pbs_runtime::block_on(command_reopen_auth_logfiles())?;
worker.log("API authentication log was rotated".to_string());
} else {
worker.log("API authentication log was not rotated".to_string());
@@ -794,7 +803,7 @@ async fn schedule_task_log_rotate() {
}
-async fn command_reopen_logfiles() -> Result<(), Error> {
+async fn command_reopen_access_logfiles() -> Result<(), Error> {
// only care about the most recent daemon instance for each, proxy & api, as other older ones
// should not respond to new requests anyway, but only finish their current one and then exit.
let sock = crate::server::our_ctrl_sock();
@@ -812,6 +821,24 @@ async fn command_reopen_logfiles() -> Result<(), Error> {
}
}
+async fn command_reopen_auth_logfiles() -> Result<(), Error> {
+ // only care about the most recent daemon instance for each, proxy & api, as other older ones
+ // should not respond to new requests anyway, but only finish their current one and then exit.
+ let sock = crate::server::our_ctrl_sock();
+ let f1 = proxmox_rest_server::send_command(sock, "{\"command\":\"api-auth-log-reopen\"}\n");
+
+ let pid = crate::server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_API_PID_FN)?;
+ let sock = crate::server::ctrl_sock_from_pid(pid);
+ let f2 = proxmox_rest_server::send_command(sock, "{\"command\":\"api-auth-log-reopen\"}\n");
+
+ match futures::join!(f1, f2) {
+ (Err(e1), Err(e2)) => Err(format_err!("reopen commands failed, proxy: {}; api: {}", e1, e2)),
+ (Err(e1), Ok(_)) => Err(format_err!("reopen commands failed, proxy: {}", e1)),
+ (Ok(_), Err(e2)) => Err(format_err!("reopen commands failed, api: {}", e2)),
+ _ => Ok(()),
+ }
+}
+
async fn run_stat_generator() {
let mut count = 0;
diff --git a/src/server/rest.rs b/src/server/rest.rs
index 2083743b..59fbb396 100644
--- a/src/server/rest.rs
+++ b/src/server/rest.rs
@@ -29,12 +29,11 @@ use proxmox::api::{
RpcEnvironmentType, UserInformation,
};
use proxmox::http_err;
-use proxmox::tools::fs::CreateOptions;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
use proxmox_rest_server::{
- ApiConfig, FileLogger, FileLogOptions, AuthError, RestEnvironment, CompressionMethod,
+ ApiConfig, FileLogger, AuthError, RestEnvironment, CompressionMethod,
extract_cookie, normalize_uri_path,
};
use proxmox_rest_server::formatter::*;
@@ -200,22 +199,6 @@ fn log_response(
}
}
-pub fn auth_logger() -> Result<FileLogger, Error> {
- let backup_user = pbs_config::backup_user()?;
-
- let file_opts = CreateOptions::new()
- .owner(backup_user.uid)
- .group(backup_user.gid);
-
- let logger_options = FileLogOptions {
- append: true,
- prefix_time: true,
- file_opts,
- ..Default::default()
- };
- FileLogger::new(pbs_buildcfg::API_AUTH_LOG_FN, logger_options)
-}
-
fn get_proxied_peer(headers: &HeaderMap) -> Option<std::net::SocketAddr> {
lazy_static! {
static ref RE: Regex = Regex::new(r#"for="([^"]+)""#).unwrap();
@@ -272,7 +255,7 @@ impl tower_service::Service<Request<Body>> for ApiService {
.body(err.into())?
}
};
- let logger = config.get_file_log();
+ let logger = config.get_access_log();
log_response(logger, &peer, method, &path, &response, user_agent);
Ok(response)
}
@@ -631,7 +614,7 @@ async fn handle_request(
}
let env_type = api.env_type();
- let mut rpcenv = RestEnvironment::new(env_type);
+ let mut rpcenv = RestEnvironment::new(env_type, Arc::clone(&api));
rpcenv.set_client_ip(Some(*peer));
@@ -675,11 +658,8 @@ async fn handle_request(
format_err!("no authentication credentials provided.")
}
};
- let peer = peer.ip();
- auth_logger()?.log(format!(
- "authentication failure; rhost={} msg={}",
- peer, err
- ));
+ // fixme: log Username??
+ rpcenv.log_failed_auth(None, &err.to_string());
// always delay unauthorized calls by 3 seconds (from start of request)
let err = http_err!(UNAUTHORIZED, "authentication failed - {}", err);
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 13/16] move src/server/rest.rs to proxmox-rest-server crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (11 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 12/16] rest server: cleanup auth-log handling Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 14/16] move proxmox_restore_daemon code into extra crate Dietmar Maurer
` (3 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
proxmox-rest-server/Cargo.toml | 4 ++++
proxmox-rest-server/src/lib.rs | 3 +++
{src/server => proxmox-rest-server/src}/rest.rs | 6 +++---
src/bin/proxmox-backup-api.rs | 3 +--
src/bin/proxmox-backup-proxy.rs | 3 +--
src/bin/proxmox-restore-daemon.rs | 4 +---
src/server/h2service.rs | 2 +-
src/server/mod.rs | 3 ---
8 files changed, 14 insertions(+), 14 deletions(-)
rename {src/server => proxmox-rest-server/src}/rest.rs (99%)
diff --git a/proxmox-rest-server/Cargo.toml b/proxmox-rest-server/Cargo.toml
index c19dfb0b..2f740e67 100644
--- a/proxmox-rest-server/Cargo.toml
+++ b/proxmox-rest-server/Cargo.toml
@@ -16,9 +16,13 @@ libc = "0.2"
log = "0.4"
nix = "0.19.1"
percent-encoding = "2.1"
+regex = "1.2"
serde = { version = "1.0", features = [] }
serde_json = "1.0"
tokio = { version = "1.6", features = ["signal", "process"] }
+tokio-openssl = "0.6.1"
+tower-service = "0.3.0"
+url = "2.1"
proxmox = { version = "0.13.3", features = [ "router"] }
diff --git a/proxmox-rest-server/src/lib.rs b/proxmox-rest-server/src/lib.rs
index 55a10ca6..2f29f9cd 100644
--- a/proxmox-rest-server/src/lib.rs
+++ b/proxmox-rest-server/src/lib.rs
@@ -26,6 +26,9 @@ pub use file_logger::{FileLogger, FileLogOptions};
mod api_config;
pub use api_config::ApiConfig;
+mod rest;
+pub use rest::{RestServer, handle_api_request};
+
pub enum AuthError {
Generic(Error),
NoData,
diff --git a/src/server/rest.rs b/proxmox-rest-server/src/rest.rs
similarity index 99%
rename from src/server/rest.rs
rename to proxmox-rest-server/src/rest.rs
index 59fbb396..dde47b55 100644
--- a/src/server/rest.rs
+++ b/proxmox-rest-server/src/rest.rs
@@ -32,11 +32,11 @@ use proxmox::http_err;
use pbs_tools::compression::{DeflateEncoder, Level};
use pbs_tools::stream::AsyncReaderStream;
-use proxmox_rest_server::{
+
+use crate::{
ApiConfig, FileLogger, AuthError, RestEnvironment, CompressionMethod,
- extract_cookie, normalize_uri_path,
+ extract_cookie, normalize_uri_path, formatter::*,
};
-use proxmox_rest_server::formatter::*;
extern "C" {
fn tzset();
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index 7653f412..9ad10260 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -10,12 +10,11 @@ use proxmox::api::RpcEnvironmentType;
use proxmox::tools::fs::CreateOptions;
use pbs_tools::auth::private_auth_key;
-use proxmox_rest_server::ApiConfig;
+use proxmox_rest_server::{ApiConfig, RestServer};
use proxmox_backup::server::{
self,
auth::default_api_auth,
- rest::*,
};
use proxmox_rest_server::daemon;
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 5679de43..6a8736e2 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -19,14 +19,13 @@ use proxmox::api::RpcEnvironmentType;
use proxmox::sys::linux::socket::set_tcp_keepalive;
use proxmox::tools::fs::CreateOptions;
-use proxmox_rest_server::ApiConfig;
+use proxmox_rest_server::{ApiConfig, RestServer};
use proxmox_backup::{
backup::DataStore,
server::{
auth::default_api_auth,
WorkerTask,
- rest::*,
jobstate::{
self,
Job,
diff --git a/src/bin/proxmox-restore-daemon.rs b/src/bin/proxmox-restore-daemon.rs
index d9a8eff0..9cf7f755 100644
--- a/src/bin/proxmox-restore-daemon.rs
+++ b/src/bin/proxmox-restore-daemon.rs
@@ -21,9 +21,7 @@ use hyper::header;
use proxmox::api::RpcEnvironmentType;
use pbs_client::DEFAULT_VSOCK_PORT;
-use proxmox_rest_server::ApiConfig;
-
-use proxmox_backup::server::rest::*;
+use proxmox_rest_server::{ApiConfig, RestServer};
mod proxmox_restore_daemon;
use proxmox_restore_daemon::*;
diff --git a/src/server/h2service.rs b/src/server/h2service.rs
index 4e3024aa..41d628be 100644
--- a/src/server/h2service.rs
+++ b/src/server/h2service.rs
@@ -61,7 +61,7 @@ impl <E: RpcEnvironment + Clone> H2Service<E> {
future::ok((formatter.format_error)(err)).boxed()
}
Some(api_method) => {
- crate::server::rest::handle_api_request(
+ proxmox_rest_server::handle_api_request(
self.rpcenv.clone(), api_method, formatter, parts, body, uri_param).boxed()
}
}
diff --git a/src/server/mod.rs b/src/server/mod.rs
index 1fc57575..a7dcee67 100644
--- a/src/server/mod.rs
+++ b/src/server/mod.rs
@@ -55,9 +55,6 @@ pub use worker_task::*;
mod h2service;
pub use h2service::*;
-#[macro_use]
-pub mod rest;
-
pub mod jobstate;
mod verify_job;
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 14/16] move proxmox_restore_daemon code into extra crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (12 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 13/16] move src/server/rest.rs to proxmox-rest-server crate Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 15/16] basically a (semantic) revert of commit 991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround linkage issues from openid/curl build server stuff separate" Dietmar Maurer
` (2 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
---
Cargo.toml | 1 +
Makefile | 4 ++-
proxmox-restore-daemon/Cargo.toml | 36 +++++++++++++++++++
.../src/main.rs | 0
.../src}/proxmox_restore_daemon/api.rs | 0
.../src}/proxmox_restore_daemon/auth.rs | 0
.../src}/proxmox_restore_daemon/disk.rs | 0
.../src}/proxmox_restore_daemon/mod.rs | 0
.../src}/proxmox_restore_daemon/watchdog.rs | 0
9 files changed, 40 insertions(+), 1 deletion(-)
create mode 100644 proxmox-restore-daemon/Cargo.toml
rename src/bin/proxmox-restore-daemon.rs => proxmox-restore-daemon/src/main.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/api.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/auth.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/disk.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/mod.rs (100%)
rename {src/bin => proxmox-restore-daemon/src}/proxmox_restore_daemon/watchdog.rs (100%)
diff --git a/Cargo.toml b/Cargo.toml
index 42528a49..95588d06 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -35,6 +35,7 @@ members = [
"proxmox-backup-client",
"proxmox-backup-debug",
"proxmox-file-restore",
+ "proxmox-restore-daemon",
"pxar-bin",
]
diff --git a/Makefile b/Makefile
index b1d1ea93..279f0523 100644
--- a/Makefile
+++ b/Makefile
@@ -47,6 +47,7 @@ SUBCRATES := \
proxmox-backup-client \
proxmox-backup-debug \
proxmox-file-restore \
+ proxmox-restore-daemon \
pxar-bin
ifeq ($(BUILD_MODE), release)
@@ -189,11 +190,12 @@ $(COMPILED_BINS) $(COMPILEDIR)/dump-catalog-shell-cli $(COMPILEDIR)/docgen: .do-
--package pbs-tape \
--bin pmt \
--bin pmtx \
+ --package proxmox-restore-daemon \
+ --bin proxmox-restore-daemon \
--package proxmox-backup \
--bin dump-catalog-shell-cli \
--bin proxmox-daily-update \
--bin proxmox-file-restore \
- --bin proxmox-restore-daemon \
--bin proxmox-tape \
--bin sg-tape-cmd
touch "$@"
diff --git a/proxmox-restore-daemon/Cargo.toml b/proxmox-restore-daemon/Cargo.toml
new file mode 100644
index 00000000..c525dc99
--- /dev/null
+++ b/proxmox-restore-daemon/Cargo.toml
@@ -0,0 +1,36 @@
+[package]
+name = "proxmox-restore-daemon"
+version = "0.1.0"
+authors = ["Proxmox Support Team <support@proxmox.com>"]
+edition = "2018"
+description = "Proxmox Restore Daemon"
+
+[dependencies]
+anyhow = "1.0"
+base64 = "0.12"
+env_logger = "0.7"
+futures = "0.3"
+http = "0.2"
+hyper = { version = "0.14", features = [ "full" ] }
+lazy_static = "1.4"
+libc = "0.2"
+log = "0.4"
+nix = "0.19.1"
+regex = "1.2"
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+tokio = { version = "1.6", features = ["parking_lot", "sync"] }
+tokio-stream = "0.1.0"
+tokio-util = { version = "0.6", features = [ "codec", "io" ] }
+
+pathpatterns = "0.1.2"
+pxar = { version = "0.10.1", features = [ "tokio-io" ] }
+
+proxmox = { version = "0.13.3", features = [ "router", "sortable-macro" ] }
+
+pbs-api-types = { path = "../pbs-api-types" }
+pbs-runtime = { path = "../pbs-runtime" }
+pbs-tools = { path = "../pbs-tools" }
+pbs-datastore = { path = "../pbs-datastore" }
+proxmox-rest-server = { path = "../proxmox-rest-server" }
+pbs-client = { path = "../pbs-client" }
diff --git a/src/bin/proxmox-restore-daemon.rs b/proxmox-restore-daemon/src/main.rs
similarity index 100%
rename from src/bin/proxmox-restore-daemon.rs
rename to proxmox-restore-daemon/src/main.rs
diff --git a/src/bin/proxmox_restore_daemon/api.rs b/proxmox-restore-daemon/src/proxmox_restore_daemon/api.rs
similarity index 100%
rename from src/bin/proxmox_restore_daemon/api.rs
rename to proxmox-restore-daemon/src/proxmox_restore_daemon/api.rs
diff --git a/src/bin/proxmox_restore_daemon/auth.rs b/proxmox-restore-daemon/src/proxmox_restore_daemon/auth.rs
similarity index 100%
rename from src/bin/proxmox_restore_daemon/auth.rs
rename to proxmox-restore-daemon/src/proxmox_restore_daemon/auth.rs
diff --git a/src/bin/proxmox_restore_daemon/disk.rs b/proxmox-restore-daemon/src/proxmox_restore_daemon/disk.rs
similarity index 100%
rename from src/bin/proxmox_restore_daemon/disk.rs
rename to proxmox-restore-daemon/src/proxmox_restore_daemon/disk.rs
diff --git a/src/bin/proxmox_restore_daemon/mod.rs b/proxmox-restore-daemon/src/proxmox_restore_daemon/mod.rs
similarity index 100%
rename from src/bin/proxmox_restore_daemon/mod.rs
rename to proxmox-restore-daemon/src/proxmox_restore_daemon/mod.rs
diff --git a/src/bin/proxmox_restore_daemon/watchdog.rs b/proxmox-restore-daemon/src/proxmox_restore_daemon/watchdog.rs
similarity index 100%
rename from src/bin/proxmox_restore_daemon/watchdog.rs
rename to proxmox-restore-daemon/src/proxmox_restore_daemon/watchdog.rs
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 15/16] basically a (semantic) revert of commit 991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround linkage issues from openid/curl build server stuff separate"
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (13 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 14/16] move proxmox_restore_daemon code into extra crate Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 16/16] worker_state: move tasktype() code to src/api2/node/tasks.rs Dietmar Maurer
2021-09-21 7:37 ` [pbs-devel] applied-series: [PATCH proxmox-backup v2 00/16] move rest server into extra crate Thomas Lamprecht
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel; +Cc: Dietmar Maurer, Thomas Lamprecht
This is no longer required because we moved proxmox_restore_daemon
code into extra crate (previous commit)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
Makefile | 5 ++---
src/api2/access/mod.rs | 12 ++----------
2 files changed, 4 insertions(+), 13 deletions(-)
diff --git a/Makefile b/Makefile
index 279f0523..8a304469 100644
--- a/Makefile
+++ b/Makefile
@@ -171,12 +171,11 @@ cargo-build:
$(COMPILED_BINS) $(COMPILEDIR)/dump-catalog-shell-cli $(COMPILEDIR)/docgen: .do-cargo-build
.do-cargo-build:
- RUSTFLAGS="--cfg openid" $(CARGO) build $(CARGO_BUILD_ARGS) \
+ $(CARGO) build $(CARGO_BUILD_ARGS) \
--bin proxmox-backup-api \
--bin proxmox-backup-proxy \
--bin proxmox-backup-manager \
- --bin docgen
- $(CARGO) build $(CARGO_BUILD_ARGS) \
+ --bin docgen \
--package proxmox-backup-banner \
--bin proxmox-backup-banner \
--package proxmox-backup-client \
diff --git a/src/api2/access/mod.rs b/src/api2/access/mod.rs
index 48a90d1b..52963d3c 100644
--- a/src/api2/access/mod.rs
+++ b/src/api2/access/mod.rs
@@ -27,13 +27,11 @@ use crate::config::tfa::TfaChallenge;
pub mod acl;
pub mod domain;
+pub mod openid;
pub mod role;
pub mod tfa;
pub mod user;
-#[cfg(openid)]
-pub mod openid;
-
#[allow(clippy::large_enum_variant)]
enum AuthResult {
/// Successful authentication which does not require a new ticket.
@@ -413,12 +411,6 @@ pub fn list_permissions(
Ok(map)
}
-#[cfg(openid)]
-const OPENID_ROUTER: &Router = &openid::ROUTER;
-
-#[cfg(not(openid))]
-const OPENID_ROUTER: &Router = &Router::new();
-
#[sortable]
const SUBDIRS: SubdirMap = &sorted!([
("acl", &acl::ROUTER),
@@ -428,7 +420,7 @@ const SUBDIRS: SubdirMap = &sorted!([
&Router::new().get(&API_METHOD_LIST_PERMISSIONS)
),
("ticket", &Router::new().post(&API_METHOD_CREATE_TICKET)),
- ("openid", &OPENID_ROUTER),
+ ("openid", &openid::ROUTER),
("domains", &domain::ROUTER),
("roles", &role::ROUTER),
("users", &user::ROUTER),
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 16/16] worker_state: move tasktype() code to src/api2/node/tasks.rs
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (14 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 15/16] basically a (semantic) revert of commit 991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround linkage issues from openid/curl build server stuff separate" Dietmar Maurer
@ 2021-09-21 5:58 ` Dietmar Maurer
2021-09-21 7:37 ` [pbs-devel] applied-series: [PATCH proxmox-backup v2 00/16] move rest server into extra crate Thomas Lamprecht
16 siblings, 0 replies; 18+ messages in thread
From: Dietmar Maurer @ 2021-09-21 5:58 UTC (permalink / raw)
To: pbs-devel
Because this is API related code, and only used there.
---
src/api2/node/tasks.rs | 11 ++++++++++-
src/server/worker_task.rs | 11 +----------
2 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/src/api2/node/tasks.rs b/src/api2/node/tasks.rs
index 9aaf6f1a..7066f889 100644
--- a/src/api2/node/tasks.rs
+++ b/src/api2/node/tasks.rs
@@ -116,6 +116,15 @@ fn check_task_access(auth_id: &Authid, upid: &UPID) -> Result<(), Error> {
}
}
+pub fn tasktype(state: &TaskState) -> TaskStateType {
+ match state {
+ TaskState::OK { .. } => TaskStateType::OK,
+ TaskState::Unknown { .. } => TaskStateType::Unknown,
+ TaskState::Error { .. } => TaskStateType::Error,
+ TaskState::Warning { .. } => TaskStateType::Warning,
+ }
+}
+
#[api(
input: {
properties: {
@@ -492,7 +501,7 @@ pub fn list_tasks(
(Some(_), _) if running => continue,
(Some(crate::server::TaskState::OK { .. }), _) if errors => continue,
(Some(state), Some(filters)) => {
- if !filters.contains(&state.tasktype()) {
+ if !filters.contains(&tasktype(state)) {
continue;
}
},
diff --git a/src/server/worker_task.rs b/src/server/worker_task.rs
index c62ba7df..565ce154 100644
--- a/src/server/worker_task.rs
+++ b/src/server/worker_task.rs
@@ -18,7 +18,7 @@ use proxmox::tools::fs::{create_path, replace_file, CreateOptions};
use pbs_buildcfg;
use pbs_tools::logrotate::{LogRotate, LogRotateFiles};
-use pbs_api_types::{Authid, TaskStateType, UPID};
+use pbs_api_types::{Authid, UPID};
use pbs_config::{open_backup_lockfile, BackupLockGuard};
use proxmox_rest_server::{CommandoSocket, FileLogger, FileLogOptions};
@@ -243,15 +243,6 @@ impl TaskState {
}
}
- pub fn tasktype(&self) -> TaskStateType {
- match self {
- TaskState::OK { .. } => TaskStateType::OK,
- TaskState::Unknown { .. } => TaskStateType::Unknown,
- TaskState::Error { .. } => TaskStateType::Error,
- TaskState::Warning { .. } => TaskStateType::Warning,
- }
- }
-
fn result_text(&self) -> String {
match self {
TaskState::Error { message, .. } => format!("TASK ERROR: {}", message),
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pbs-devel] applied-series: [PATCH proxmox-backup v2 00/16] move rest server into extra crate
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
` (15 preceding siblings ...)
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 16/16] worker_state: move tasktype() code to src/api2/node/tasks.rs Dietmar Maurer
@ 2021-09-21 7:37 ` Thomas Lamprecht
16 siblings, 0 replies; 18+ messages in thread
From: Thomas Lamprecht @ 2021-09-21 7:37 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Dietmar Maurer
On 21.09.21 07:58, Dietmar Maurer wrote:
> Changes in v2:
>
> - renamed the crate from "pbs-server" into "proxmox-rest-server"
> - depend on proxmox "0.13.3"
> - worker_state: move tasktype() code to src/api2/node/tasks.rs
> - dependency fixes for proxmox_restore_daemon crate (fabian)
>
> Dietmar Maurer (16):
> start new proxmox-rest-server workspace
> move ApiConfig, FileLogger and CommandoSocket to proxmox-rest-server
> workspace
> move src/tools/daemon.rs to proxmox-rest-server workspace
> move src/server/environment.rs to proxmox-rest-server crate
> move src/server/formatter.rs to proxmox-rest-server crate
> move src/tools/compression.rs to proxmox-rest-server crate
> move normalize_uri_path and extract_cookie to proxmox-rest-server
> crate
> rest server: simplify get_index() method signature
> make get_index and ApiConfig property (callback)
> rest server: return UserInformation from ApiAuth::check_auth
> rest server: do not use pbs_api_types::Authid
> rest server: cleanup auth-log handling
> move src/server/rest.rs to proxmox-rest-server crate
> move proxmox_restore_daemon code into extra crate
> basically a (semantic) revert of commit
> 991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround
> linkage issues from openid/curl build server stuff separate"
> worker_state: move tasktype() code to src/api2/node/tasks.rs
>
applied whole series, thanks!
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2021-09-21 7:39 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-21 5:58 [pbs-devel] [PATCH proxmox-backup v2 00/16] move rest server into extra crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 01/16] start new proxmox-rest-server workspace Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 02/16] move ApiConfig, FileLogger and CommandoSocket to " Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 03/16] move src/tools/daemon.rs " Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 04/16] move src/server/environment.rs to proxmox-rest-server crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 05/16] move src/server/formatter.rs " Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 06/16] move src/tools/compression.rs " Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 07/16] move normalize_uri_path and extract_cookie " Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 08/16] rest server: simplify get_index() method signature Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 09/16] make get_index and ApiConfig property (callback) Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 10/16] rest server: return UserInformation from ApiAuth::check_auth Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 11/16] rest server: do not use pbs_api_types::Authid Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 12/16] rest server: cleanup auth-log handling Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 13/16] move src/server/rest.rs to proxmox-rest-server crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 14/16] move proxmox_restore_daemon code into extra crate Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 15/16] basically a (semantic) revert of commit 991be99c37c6f55f43a3d9a2c54edb2a8dc6d4f2 "buildsys: workaround linkage issues from openid/curl build server stuff separate" Dietmar Maurer
2021-09-21 5:58 ` [pbs-devel] [PATCH proxmox-backup v2 16/16] worker_state: move tasktype() code to src/api2/node/tasks.rs Dietmar Maurer
2021-09-21 7:37 ` [pbs-devel] applied-series: [PATCH proxmox-backup v2 00/16] move rest server into extra crate Thomas Lamprecht
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal