From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
Cc: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com,
pmg-devel@lists.proxmox.com
Subject: [pmg-devel] applied: [pbs-devel] [PATCH backup v4] make tasklog downloadable in the backup server backend
Date: Thu, 24 Nov 2022 11:26:17 +0100 [thread overview]
Message-ID: <20221124102617.6bdrqcrjj5zyhi6o@casey.proxmox.com> (raw)
In-Reply-To: <20221123145210.592981-2-d.tschlatscher@proxmox.com>
applied with a cleanup followup commit, see below
On Wed, Nov 23, 2022 at 03:52:07PM +0100, Daniel Tschlatscher wrote:
> The read_tasklog API call now stream the whole log file if the query
> parameter 'download' is set to true. If the limit parameter is set to
> 0, all lines in the tasklog will be returned in json format.
>
> To make a file stream and a json response in the same API call work, I
> had to use one of the lower level apimethod types from the
> proxmox-router. Therefore, the routing declarations and parameter
> schemas have been changed accordingly.
>
> Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
> ---
> I sent this separately, as the other 3 patches in this version do not
> concern the backup patch in any way.
>
> Changes from v3:
> * API parameter 'download' for streaming the file directly
> * option 'limit=0' returns the whole task log
> * read_task_log_lines() function is inline in the code again as it's
> the only occurrence and doesn't add that much readability
>
> src/api2/node/tasks.rs | 209 +++++++++++++++++++++++++----------------
> 1 file changed, 129 insertions(+), 80 deletions(-)
>
> diff --git a/src/api2/node/tasks.rs b/src/api2/node/tasks.rs
> index cbd0894e..258c3e86 100644
> --- a/src/api2/node/tasks.rs
> +++ b/src/api2/node/tasks.rs
> @@ -2,10 +2,18 @@ use std::fs::File;
> use std::io::{BufRead, BufReader};
>
> use anyhow::{bail, Error};
> +use futures::FutureExt;
> +use http::request::Parts;
> +use http::{header, Response, StatusCode};
> +use hyper::Body;
> +use proxmox_async::stream::AsyncReaderStream;
> use serde_json::{json, Value};
>
> -use proxmox_router::{list_subdirs_api_method, Permission, Router, RpcEnvironment, SubdirMap};
> -use proxmox_schema::api;
> +use proxmox_router::{
> + list_subdirs_api_method, ApiHandler, ApiMethod, ApiResponseFuture, Permission, Router,
> + RpcEnvironment, SubdirMap,
> +};
> +use proxmox_schema::{api, BooleanSchema, IntegerSchema, ObjectSchema, Schema};
> use proxmox_sys::sortable;
>
> use pbs_api_types::{
> @@ -19,6 +27,27 @@ use crate::api2::pull::check_pull_privs;
> use pbs_config::CachedUserInfo;
> use proxmox_rest_server::{upid_log_path, upid_read_status, TaskListInfoIterator, TaskState};
>
> +pub const START_PARAM_SCHEMA: Schema =
> + IntegerSchema::new("Start at this line when reading the tasklog")
> + .minimum(0)
> + .default(0)
> + .schema();
> +
> +pub const LIMIT_PARAM_SCHEMA: Schema =
> + IntegerSchema::new("The amount of lines to read from the tasklog. Setting this parameter to 0 will return all lines until the end of the file.")
^ Please split lines longer than 100 chars, like:
IntegerSchema::new(
"The amount of lines to read from the tasklog. \
Setting this parameter to 0 will return all lines until the end of the file."
)
(...)
> + if download {
> + if !param["start"].is_null()
> + || !param["limit"].is_null()
> + || !param["test-status"].is_null()
> + {
> + bail!("Parameter 'download' cannot be used with other parameters");
> + }
>
> - for line in BufReader::new(file).lines() {
> - match line {
> - Ok(line) => {
> - count += 1;
> - if count < start {
> - continue;
> - };
> - if limit == 0 {
> - continue;
> - };
> + let header_disp = format!("attachment; filename={}", &upid.to_string());
> + let stream = AsyncReaderStream::new(tokio::fs::File::open(path).await?);
> +
> + Ok(Response::builder()
> + .status(StatusCode::OK)
> + .header(header::CONTENT_TYPE, "text/plain")
> + .header(header::CONTENT_DISPOSITION, &header_disp)
> + .body(Body::wrap_stream(stream))
> + .unwrap())
^ This could just have a `return` in front of it, so the remaining code
isn't that much indented.
> + } else {
> + let start = param["start"].as_u64().unwrap_or(0);
> + let mut limit = param["limit"].as_u64().unwrap_or(50);
> + let test_status = param["test-status"].as_bool().unwrap_or(false);
> +
> + let file = File::open(path)?;
> +
> + let mut count: u64 = 0;
> + let mut lines: Vec<Value> = vec![];
> + let read_until_end = if limit == 0 { true } else { false };
^ booleans are already booleans, please avoid this construct in the
future.
clippy can catch this btw.
next prev parent reply other threads:[~2022-11-24 10:26 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-23 14:52 [pmg-devel] [PATCH backup/pmg-api/manager/common v4] fix: #3971 Tasklog download button Daniel Tschlatscher
2022-11-23 14:52 ` [pmg-devel] [PATCH backup v4] make tasklog downloadable in the backup server backend Daniel Tschlatscher
2022-11-24 10:26 ` Wolfgang Bumiller [this message]
2022-11-23 14:52 ` [pmg-devel] [PATCH pmg-api v4] make tasklog downloadable in the PMG backend Daniel Tschlatscher
2022-11-23 14:52 ` [pmg-devel] [PATCH manager v4] make task log downloadable in the PVE manager backend Daniel Tschlatscher
2022-11-23 14:52 ` [pmg-devel] [PATCH common v4] return whole log file if limit is 0 Daniel Tschlatscher
2022-11-24 16:24 ` [pmg-devel] applied: [pve-devel] " Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221124102617.6bdrqcrjj5zyhi6o@casey.proxmox.com \
--to=w.bumiller@proxmox.com \
--cc=d.tschlatscher@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
--cc=pmg-devel@lists.proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox