[pve-devel] OpenAPI converter (similar to JSONSchema generator)

[pve-devel] OpenAPI converter (similar to JSONSchema generator)

[Erik Hollensbe]

The specification is here and has similar semantics and goals to
jsonschema, just really good codegen support:
https://swagger.io/specification/

What I am proposing is a module that sits alongside the JSONSchema package
and generates an openapi.json/swagger.json on http request, so that the
client can either be dynamically configured or generated from a request to
a server. It could also be served on proxmox's website for simpler
generation.

It looks like this is doable with the separation of concerns in JSONSchema
and RESTHandler, since all the former does is extract data from the latter,
unless I am misreading things. A swagger/openapi generator would do
something similar. Swagger is just JSON as well, so there's no magic
formatting or parsing that needs to take place.

I am comfortable (but rusty) in perl and can read your json schema code
fine, and am willing to make the patches to generate the specification
assuming this is OK and acceptable to the powers that be.

The advantages are a healthy ecosystem of code generators (
https://github.com/OpenAPITools/openapi-generator) and also documentation
tools, like this one: https://redocly.github.io/redoc/.

I think the package could be delivered in a few weeks assuming a
development environment is easy enough to get going.

Thanks for your time,

-Erik

Re: [pve-devel] OpenAPI converter (similar to JSONSchema generator)

[Thomas Lamprecht]

Hi,


first, thanks for your interest in working on and improving Proxmox VE!

On 17/12/2020 05:45, Erik Hollensbe wrote:
> The specification is here and has similar semantics and goals to
> jsonschema, just really good codegen support:
> https://swagger.io/specification/
> 
> What I am proposing is a module that sits alongside the JSONSchema package
> and generates an openapi.json/swagger.json on http request, so that the
> client can either be dynamically configured or generated from a request to
> a server. It could also be served on proxmox's website for simpler
> generation.

You mean, you'd like to add a new API endpoint returning the schema in
openAPI format?

> 
> It looks like this is doable with the separation of concerns in JSONSchema
> and RESTHandler, since all the former does is extract data from the latter,
> unless I am misreading things. A swagger/openapi generator would do
> something similar. Swagger is just JSON as well, so there's no magic
> formatting or parsing that needs to take place.

JSONSchema and openAPI are closely related, so it should not be really hard
to do, albeit, we're not 100% JSONSchema compatible, meaning that we do not
support all of it and possible use some things which it has no support for
(on the latter I'd need to re-check closely though).

The code dumping it should live along side of JSONSchema in pve-common, but
the actual API endpoint (if that is what you want) has to go in pve-manager,
as only there we have all dependencies available for actual use.

> 
> I am comfortable (but rusty) in perl and can read your json schema code
> fine, and am willing to make the patches to generate the specification
> assuming this is OK and acceptable to the powers that be.

We have a wiki giving some pointers about build environment, code-style,
CLA, and sending patches - maybe it helps checking it out:

https://pve.proxmox.com/wiki/Developer_Documentation

> 
> The advantages are a healthy ecosystem of code generators (
> https://github.com/OpenAPITools/openapi-generator) and also documentation
> tools, like this one: https://redocly.github.io/redoc/.
> 
> I think the package could be delivered in a few weeks assuming a
> development environment is easy enough to get going.
> 

See the following readme for some rough instructions, note that just for the
perl development lots 
https://git.proxmox.com/?p=pve-common.git;a=blob_plain;f=README.dev;hb=HEAD

cheers,
Thomas

Re: [pve-devel] OpenAPI converter (similar to JSONSchema generator)

[Erik Hollensbe]

Thanks for your assistance! I will look into these things over the weekend.

As for the swagger endpoint, yes, more or less, I was going to assume there
could be a /api2/swagger.json or similar route that could resolve to the
API definition.

On Fri, Dec 18, 2020 at 2:02 AM Thomas Lamprecht <t.lamprecht@proxmox.com>
wrote:

> Hi,
>
>
> first, thanks for your interest in working on and improving Proxmox VE!
>
> On 17/12/2020 05:45, Erik Hollensbe wrote:
> > The specification is here and has similar semantics and goals to
> > jsonschema, just really good codegen support:
> > https://swagger.io/specification/
> >
> > What I am proposing is a module that sits alongside the JSONSchema
> package
> > and generates an openapi.json/swagger.json on http request, so that the
> > client can either be dynamically configured or generated from a request
> to
> > a server. It could also be served on proxmox's website for simpler
> > generation.
>
> You mean, you'd like to add a new API endpoint returning the schema in
> openAPI format?
>
> >
> > It looks like this is doable with the separation of concerns in
> JSONSchema
> > and RESTHandler, since all the former does is extract data from the
> latter,
> > unless I am misreading things. A swagger/openapi generator would do
> > something similar. Swagger is just JSON as well, so there's no magic
> > formatting or parsing that needs to take place.
>
> JSONSchema and openAPI are closely related, so it should not be really hard
> to do, albeit, we're not 100% JSONSchema compatible, meaning that we do not
> support all of it and possible use some things which it has no support for
> (on the latter I'd need to re-check closely though).
>
> The code dumping it should live along side of JSONSchema in pve-common, but
> the actual API endpoint (if that is what you want) has to go in
> pve-manager,
> as only there we have all dependencies available for actual use.
>
> >
> > I am comfortable (but rusty) in perl and can read your json schema code
> > fine, and am willing to make the patches to generate the specification
> > assuming this is OK and acceptable to the powers that be.
>
> We have a wiki giving some pointers about build environment, code-style,
> CLA, and sending patches - maybe it helps checking it out:
>
> https://pve.proxmox.com/wiki/Developer_Documentation
>
> >
> > The advantages are a healthy ecosystem of code generators (
> > https://github.com/OpenAPITools/openapi-generator) and also
> documentation
> > tools, like this one: https://redocly.github.io/redoc/.
> >
> > I think the package could be delivered in a few weeks assuming a
> > development environment is easy enough to get going.
> >
>
> See the following readme for some rough instructions, note that just for
> the
> perl development lots
> https://git.proxmox.com/?p=pve-common.git;a=blob_plain;f=README.dev;hb=HEAD
>
> cheers,
> Thomas
>
>

Re: [pve-devel] OpenAPI converter (similar to JSONSchema generator)

[Thomas Lamprecht]

On 18/12/2020 12:27, Erik Hollensbe wrote:
> Thanks for your assistance! I will look into these things over the weekend.
> 
> As for the swagger endpoint, yes, more or less, I was going to assume there could be a /api2/swagger.json or similar route that could resolve to the API definition.
> 

I'd mount it under `/schema/openapi` or maybe `/meta/openapi` as FWICT,
this does not needs to be swagger specific, they just understand the
openAPI spec.

The /api2/<formatter>/ then gets always prefixed, <formatter> can be `json`, `html`
(bootstrap backed API "debug" helper) or `extjs` (nested format tailored to our web
interface framework).

For generation I'd like to have this cached in `/var/lib/pve-manager` or so, can
be updated by pve-manager through the `pve-api-updates` debian packaging trigger
we already use, this way accessing that endpoint is fast and correct as it does
not needs to dump it on demand.

Note also that we do already some API dumping and use that, the pve-docs api
viewer[0] is based on similar functionality, there we generate a JS file[1]
with is basically a JSON structure.

[0]: https://pve.proxmox.com/pve-docs/api-viewer/index.html#/nodes/{node}
[1]: https://git.proxmox.com/?p=pve-docs.git;a=blob;f=api-viewer/apidata.js;h=a8f41de3f084219727581ae7a59ab6f176756890;hb=60077fc155c73905009c57874bffca1ec1fff49f 


> On Fri, Dec 18, 2020 at 2:02 AM Thomas Lamprecht <t.lamprecht@proxmox.com <mailto:t.lamprecht@proxmox.com>> wrote:
> 
>     Hi,
> 
> 
>     first, thanks for your interest in working on and improving Proxmox VE!
> 
>     On 17/12/2020 05:45, Erik Hollensbe wrote:
>     > The specification is here and has similar semantics and goals to
>     > jsonschema, just really good codegen support:
>     > https://swagger.io/specification/ <https://swagger.io/specification/>
>     >
>     > What I am proposing is a module that sits alongside the JSONSchema package
>     > and generates an openapi.json/swagger.json on http request, so that the
>     > client can either be dynamically configured or generated from a request to
>     > a server. It could also be served on proxmox's website for simpler
>     > generation.
> 
>     You mean, you'd like to add a new API endpoint returning the schema in
>     openAPI format?
> 
>     >
>     > It looks like this is doable with the separation of concerns in JSONSchema
>     > and RESTHandler, since all the former does is extract data from the latter,
>     > unless I am misreading things. A swagger/openapi generator would do
>     > something similar. Swagger is just JSON as well, so there's no magic
>     > formatting or parsing that needs to take place.
> 
>     JSONSchema and openAPI are closely related, so it should not be really hard
>     to do, albeit, we're not 100% JSONSchema compatible, meaning that we do not
>     support all of it and possible use some things which it has no support for
>     (on the latter I'd need to re-check closely though).
> 
>     The code dumping it should live along side of JSONSchema in pve-common, but
>     the actual API endpoint (if that is what you want) has to go in pve-manager,
>     as only there we have all dependencies available for actual use.
> 
>     >
>     > I am comfortable (but rusty) in perl and can read your json schema code
>     > fine, and am willing to make the patches to generate the specification
>     > assuming this is OK and acceptable to the powers that be.
> 
>     We have a wiki giving some pointers about build environment, code-style,
>     CLA, and sending patches - maybe it helps checking it out:
> 
>     https://pve.proxmox.com/wiki/Developer_Documentation <https://pve.proxmox.com/wiki/Developer_Documentation>
> 
>     >
>     > The advantages are a healthy ecosystem of code generators (
>     > https://github.com/OpenAPITools/openapi-generator <https://github.com/OpenAPITools/openapi-generator>) and also documentation
>     > tools, like this one: https://redocly.github.io/redoc/ <https://redocly.github.io/redoc/>.
>     >
>     > I think the package could be delivered in a few weeks assuming a
>     > development environment is easy enough to get going.
>     >
> 
>     See the following readme for some rough instructions, note that just for the
>     perl development lots
>     https://git.proxmox.com/?p=pve-common.git;a=blob_plain;f=README.dev;hb=HEAD <https://git.proxmox.com/?p=pve-common.git;a=blob_plain;f=README.dev;hb=HEAD>
> 
>     cheers,
>     Thomas
>