[pve-devel] [PATCH manager 1/2] fix: api2: add return type to nodes/{node}/execute endpoint

[pve-devel] [PATCH manager 1/2] fix: api2: add return type to nodes/{node}/execute endpoint

[Stefan Sterz]

since this was missing a proper return type definition the api viewer
couldn't display the endpoint (`retinfs.items` was undefined). also
the `pvesh` command would complain that it cannot properly format the
return type because the variable `$item_type` in `CLIFormatter.pm` was
not defined.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
---
 PVE/API2/Nodes.pm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 655493a3..ef946301 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -438,8 +438,9 @@ __PACKAGE__->register_method({
     },
     returns => {
 	type => 'array',
-	properties => {
-
+	items => {
+	    type => "object",
+	    properties => {},
 	},
     },
     code => sub {
-- 
2.30.2

[pve-devel] [PATCH manager 2/2] api2: use JSONSchema to validate commands for "nodes/{node}/execute"

[Stefan Sterz]

this also makes it more explicit what the different values should be

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
---
not sure how sensible this is because most of the information here
won't show up in the api viewer. i couldn't figure out how to make it
show up and not make breaking changes to the endpoint or change how
the api definition hash is handled.

fabian gruenbichler and i also discussed off list that this is kind of
redudant functionality as you can likely just call multiple api
endpoints from whatever script/program/client you are using anyway
instead of using this limited and flawed endpoint. perhaps the was a
reason for this before tokens existed?

so maybe we should either drop it in pve 8 or let it accept proper
json not just "a string that when parsed as json needs to be of this
format". i.e. accept an array of objects here instead of a string.

it has also confused users in the past already, e.g. see:
https://forum.proxmox.com/threads/execute-command-in-node-with-api.112290/

 PVE/API2/Nodes.pm | 51 +++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 45 insertions(+), 6 deletions(-)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index ef946301..5cc9111d 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -66,6 +66,49 @@ eval {
 
 use base qw(PVE::RESTHandler);
 
+PVE::JSONSchema::register_format('pve-command-batch', \&verify_command_batch);
+sub verify_command_batch {
+    my ($value, $noerr) = @_;
+    my $commands = eval { decode_json($value); };
+
+    return undef if $noerr && $@;
+    die "commands param did not contain valid JSON: $@" if $@;
+
+    eval {
+	PVE::JSONSchema::validate($commands, {
+	    description => "An array of objects describing endpoints, methods and arguments.",
+	    type => "array",
+	    items => {
+		type => "object",
+		properties => {
+		    path => {
+			description => "A relative path to an API endpoint on this node.",
+			type => "string",
+			optional => 0,
+		    },
+		    method => {
+			description => "A method related to the API endpoint (GET, POST etc.).",
+			type => "string",
+			pattern => "(GET|POST|PUT|DELETE)",
+			optional => 0,
+		    },
+		    args => {
+			description => "A set of parameter names and their values.",
+			type => "object",
+			optional => 1,
+		    },
+		},
+	    }
+	});
+   };
+
+   return $commands if !$@;
+
+   return undef if $noerr;
+
+   die "commands is not a valid array of commands: $@";
+}
+
 __PACKAGE__->register_method ({
     subclass => "PVE::API2::Qemu",
     path => 'qemu',
@@ -433,6 +476,7 @@ __PACKAGE__->register_method({
 	    commands => {
 		description => "JSON encoded array of commands.",
 		type => "string",
+		format => "pve-command-batch",
 	    }
 	},
     },
@@ -449,16 +493,11 @@ __PACKAGE__->register_method({
 
 	my $rpcenv = PVE::RPCEnvironment::get();
 	my $user = $rpcenv->get_user();
-
+	# just parse the json again, it should already be validated
 	my $commands = eval { decode_json($param->{commands}); };
 
-	die "commands param did not contain valid JSON: $@" if $@;
-	die "commands is not an array" if ref($commands) ne "ARRAY";
-
         foreach my $cmd (@$commands) {
 	    eval {
-		die "$cmd is not a valid command" if (ref($cmd) ne "HASH" || !$cmd->{path} || !$cmd->{method});
-
 		$cmd->{args} //= {};
 
 		my $path = "nodes/$param->{node}/$cmd->{path}";
-- 
2.30.2

[pve-devel] applied+follow-up: [PATCH manager 2/2] api2: use JSONSchema to validate commands for "nodes/{node}/execute"

[Fabian Grünbichler]

by extracting the actual schema used in the validator into a varaible, 
and dumping the contained item properties into the verbose_description 
of the string API type we at least get a full description in the api 
viewer.

On July 27, 2022 4:56 pm, Stefan Sterz wrote:
> this also makes it more explicit what the different values should be
> 
> Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
> ---
> not sure how sensible this is because most of the information here
> won't show up in the api viewer. i couldn't figure out how to make it
> show up and not make breaking changes to the endpoint or change how
> the api definition hash is handled.
> 
> fabian gruenbichler and i also discussed off list that this is kind of
> redudant functionality as you can likely just call multiple api
> endpoints from whatever script/program/client you are using anyway
> instead of using this limited and flawed endpoint. perhaps the was a
> reason for this before tokens existed?
> 
> so maybe we should either drop it in pve 8 or let it accept proper
> json not just "a string that when parsed as json needs to be of this
> format". i.e. accept an array of objects here instead of a string.
> 
> it has also confused users in the past already, e.g. see:
> https://forum.proxmox.com/threads/execute-command-in-node-with-api.112290/
> 
>  PVE/API2/Nodes.pm | 51 +++++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 45 insertions(+), 6 deletions(-)
> 
> diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
> index ef946301..5cc9111d 100644
> --- a/PVE/API2/Nodes.pm
> +++ b/PVE/API2/Nodes.pm
> @@ -66,6 +66,49 @@ eval {
>  
>  use base qw(PVE::RESTHandler);
>  
> +PVE::JSONSchema::register_format('pve-command-batch', \&verify_command_batch);
> +sub verify_command_batch {
> +    my ($value, $noerr) = @_;
> +    my $commands = eval { decode_json($value); };
> +
> +    return undef if $noerr && $@;
> +    die "commands param did not contain valid JSON: $@" if $@;
> +
> +    eval {
> +	PVE::JSONSchema::validate($commands, {
> +	    description => "An array of objects describing endpoints, methods and arguments.",
> +	    type => "array",
> +	    items => {
> +		type => "object",
> +		properties => {
> +		    path => {
> +			description => "A relative path to an API endpoint on this node.",
> +			type => "string",
> +			optional => 0,
> +		    },
> +		    method => {
> +			description => "A method related to the API endpoint (GET, POST etc.).",
> +			type => "string",
> +			pattern => "(GET|POST|PUT|DELETE)",
> +			optional => 0,
> +		    },
> +		    args => {
> +			description => "A set of parameter names and their values.",
> +			type => "object",
> +			optional => 1,
> +		    },
> +		},
> +	    }
> +	});
> +   };
> +
> +   return $commands if !$@;
> +
> +   return undef if $noerr;
> +
> +   die "commands is not a valid array of commands: $@";
> +}
> +
>  __PACKAGE__->register_method ({
>      subclass => "PVE::API2::Qemu",
>      path => 'qemu',
> @@ -433,6 +476,7 @@ __PACKAGE__->register_method({
>  	    commands => {
>  		description => "JSON encoded array of commands.",
>  		type => "string",
> +		format => "pve-command-batch",
>  	    }
>  	},
>      },
> @@ -449,16 +493,11 @@ __PACKAGE__->register_method({
>  
>  	my $rpcenv = PVE::RPCEnvironment::get();
>  	my $user = $rpcenv->get_user();
> -
> +	# just parse the json again, it should already be validated
>  	my $commands = eval { decode_json($param->{commands}); };
>  
> -	die "commands param did not contain valid JSON: $@" if $@;
> -	die "commands is not an array" if ref($commands) ne "ARRAY";
> -
>          foreach my $cmd (@$commands) {
>  	    eval {
> -		die "$cmd is not a valid command" if (ref($cmd) ne "HASH" || !$cmd->{path} || !$cmd->{method});
> -
>  		$cmd->{args} //= {};
>  
>  		my $path = "nodes/$param->{node}/$cmd->{path}";
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> 

[pve-devel] applied: [PATCH manager 1/2] fix: api2: add return type to nodes/{node}/execute endpoint

[Fabian Grünbichler]

thanks!

On July 27, 2022 4:56 pm, Stefan Sterz wrote:
> since this was missing a proper return type definition the api viewer
> couldn't display the endpoint (`retinfs.items` was undefined). also
> the `pvesh` command would complain that it cannot properly format the
> return type because the variable `$item_type` in `CLIFormatter.pm` was
> not defined.
> 
> Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
> ---
>  PVE/API2/Nodes.pm | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
> index 655493a3..ef946301 100644
> --- a/PVE/API2/Nodes.pm
> +++ b/PVE/API2/Nodes.pm
> @@ -438,8 +438,9 @@ __PACKAGE__->register_method({
>      },
>      returns => {
>  	type => 'array',
> -	properties => {
> -
> +	items => {
> +	    type => "object",
> +	    properties => {},
>  	},
>      },
>      code => sub {
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
>