You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2020/11/18 13:33:05 UTC
[GitHub] [apisix-dashboard] liuxiran opened a new issue #825: [Proposal]: Import and Export route
liuxiran opened a new issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825
Please answer these questions before submitting your issue.
- Why do you submit this issue?
- [ ] Feature or performance improvement
Refer to https://github.com/apache/apisix-dashboard/issues/611
___
### Why
Sometimes user's apis have already defined based on the openapi specification and when they want to publish those apis on APISIX, they need to learn the concepts of our products, such as how to do parameter validation, how to do authentication, how to do path matching and so on. Actually, many users don't care how to publish a route, so If we can support importing the configuration of openapi to generate the route, that would reduce the user's use cost.
### How
- add buttons for import and export in route page
import button for upload the profile of route based on the openapi 3.0, and the export button for download it.
- manage-api parse the file, generate an apisix-compliant data structure, and write to the etcd
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] moonming commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-736275892
got it. So I think the first plan is good for me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-745109777
> * Expand OAS3.0, increase `upstream` and `service`, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].
>
> * advantages:
> when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
> we can support batch operations.
> this also can be done only as a command-line tool.
> * disadvantages:
> Users need to modify the existing OAS refer to our doc.
> `upstream` and `service` are the only necessary expansiones, if we want to support other plugins by `import` operation, we also need to define new expansion.
Based on the first plan, how to transform route configuration to OAS3.0?
After further compare the configuration of OAS3.0 and routes in APISIX, we categorized the current route fields into three parts
* explicitly matched fields are:
| route field | OAS3.0 field|
| ---- | ---- |
| name | paths[url][method]operationId |
| desc | paths[url][method]summary |
| methods | paths[url]method |
* We can do the conversion to implement the corresponding fields are:
| route field | OAS3.0 field|
| ---- | ---- |
| uris | paths.url, |
| vars | |
| plugins.key-auth | [OAS3.0 reference](https://swagger.io/docs/specification/authentication/api-keys/) |
| plugins.basic-auth | [OAS3.0 reference](https://swagger.io/docs/specification/authentication/basic-authentication//) |
| plugins.jwt-auth | [OAS3.0 reference](https://swagger.io/docs/specification/authentication/bearer-authentication/) |
| plugins.openid-connect | [OAS3.0 reference](https://swagger.io/docs/specification/authentication/openid-connect-discovery//) |
| plugins.request-validation | [header params OAS3.0 reference](https://swagger.io/docs/specification/describing-parameters/) 、[body params OAS3.0 reference](https://swagger.io/docs/specification/describing-request-body/) |
* we need to expended fields are:
_we recommend to follow the data structure of the route, only change the name of the key, We can define the first supported fields, prefixed with `x-apisix-*`._
| route field | OAS3.0 expanded field|
| ---- | ---- |
| upstream | paths[url][method]x-apisix-upstream [apisix upstream ref](https://github.com/apache/apisix/blob/master/doc/architecture-design.md#upstream) |
| plugins(except for a few plugins mentioned above) | paths[url][method]x-apisix-plugins [apisix plugins ref](https://github.com/apache/apisix/blob/master/doc/architecture-design.md#plugin) |
| hosts | paths[url][method]x-apisix-hosts |
---
we formed the first phase of development plan below:
1. **Export routes configuration as a json/yaml file in the way we expanded OAS3.0.**
BE:
new a manager-api
params: routeids & filetype(json or yaml)
returns: file
FE:
add select box to each of route items in route list
add a button `Export routes as OAS3.0` to the top of the route list
select one or more routes, and click the Export button, get the OAS3.0 file
2. **import the exported file to generate routes.**
BE:
new an API
params: file
returns: success or failed
FE:
add a button `Import routes from OAS3.0` at the top of the route list
click button, upload file
3. **Document: Write a document to describe our expansion configuration options based on OAS3.0, examples+description.**
cc @LiteSun @juzhiyuan @moonming @membphis @nic-chen @gxthrj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] moonming commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-736271095
> For the upstream info or service info which are not supported by OAS3.0, there are two feasible options of implementation scenarios:
>
> * Expand OAS3.0, increase `upstream` and `service`, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].
>
> * advantages:
> when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
> we can support batch operations.
> this also can be done only as a command-line tool.
> * disadvantages:
> Users need to modify the existing OAS refer to our doc.
> `upstream` and `service` are the only necessary expansiones, if we want to support other plugins by `import` operation, we also need to define new expansion.
>
> * Combine with existing create route page, when user create a route, they can import a OAS file, which provides parts of the route's info, and for the upstream 、 service and other plugins, they can follow the page step.
>
> * advantages:
> We don't need to expand OAS3.0.
> It can support all features supported by APISIX
> * disadvantages:
> We need to modify the current UE of create route to support this feature.
>
>
> Reference:
> [1]: https://help.aliyun.com/document_detail/88956.html?spm=a2c4g.11174283.6.564.7be17d24shSMBS
>
> what do you think?
>
> cc @juzhiyuan @gxthrj @LiteSun @nic-chen @membphis @moonming
how can the user export it if using the second plan?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran edited a comment on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran edited a comment on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-748009952
After the offline meeting, we have agreed on the following points:
1. First implement import/export based on OAS 3.0, and consider implementing other formats later
2. import and export in atomic.
Todo before development：
1. need to List ignored OAS fields when import OAS3.0 @liuxiran
2. Investigate whether golang has librarys that enables functionality.
cc @juzhiyuan @LiteSun @nic-chen @membphis @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] LiteSun commented on issue #825: [Proposal]: Import and Export route
Posted by GitBox <gi...@apache.org>.
LiteSun commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-729698685
I think this feature will save a lot of time for users who need to import a lot of APIs.
According to the above description, the workload on the front-end is relatively small, and the challenge is to convert the profile into an apisix-compliant data structure.
let's cc @nic-chen @gxthrj for more suggestions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] membphis commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
membphis commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-749993175
> cc @membphis
LGTM, thx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] gxthrj commented on issue #825: [Proposal]: Import and Export route
Posted by GitBox <gi...@apache.org>.
gxthrj commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-729820839
OpenAPI is a description document used to provide quick reading and testing. The route in APISIX is also used to manage API, there are still some differences. For example, we need to consider how to convert the plugin logic in route.
We'd better limit the import / export responsibility boundary and provide a more detailed transformation example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran closed issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran closed issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] juzhiyuan commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-733425201
Once this proposal and discussion are completed, we could remove the discuss label.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-736264421
For the upstream info or service info which are not supported by OAS3.0, there are two feasible options of implementation scenarios:
- Expand OAS3.0, increase `upstream` and `service`, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].
- advantages:
when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
we can support batch operations.
this also can be done only as a command-line tool.
- disadvantages:
Users need to modify the existing OAS refer to our doc.
`upstream` and `service` are the only necessary expansiones, if we want to support other plugins by `import` operation, we also need to define new expansion.
- Combine with existing create route page, when user create a route, they can import a OAS file, which provides parts of the route's info, and for the upstream 、 service and other plugins, they can follow the page step.
- advantages:
We don't need to expand OAS3.0.
It can support all features supported by APISIX
- disadvantages:
We need to modify the current UE of create route to support this feature.
Reference:
[1]: https://help.aliyun.com/document_detail/88956.html?spm=a2c4g.11174283.6.564.7be17d24shSMBS
what do you think?
cc @juzhiyuan @gxthrj @LiteSun @nic-chen @membphis @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-736274949
> how can the user export it if using the second plan?
with the second plan, user can only export part of the route info which supported by OAS3.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] juzhiyuan commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-749280961
as for API Documentation, here is an example for both 2.0 and 3.0.
https://editor.swagger.io/
cc @membphis
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran edited a comment on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran edited a comment on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-748009952
We just holed a offline meeting, and agreed on the following points:
1. First implement import/export based on OAS 3.0, and consider implementing other formats later
2. import and export in atomic.
Todo before development：
1. need to List ignored OAS fields when import OAS3.0 @liuxiran
2. Investigate whether golang has librarys that enables functionality.
cc @juzhiyuan @LiteSun @nic-chen @membphis @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] nic-chen commented on issue #825: [Proposal]: Import and Export route
Posted by GitBox <gi...@apache.org>.
nic-chen commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-729734103
@liuxiran @LiteSun
I think we need a detailed example of the imported and exported data so that everyone can understand the details more easily
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-769037033
all the related PR have been merged, close it now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] juzhiyuan commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-733424963
Here is the proposal for Route Export & Import feature but still under discussion.
ping @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-748009952
We just holed a offline meeting, and agreed on the following points:
1. First implement import/outport based on OAS 3.0, and consider implementing other formats later
2. import and export in atomic.
Todo before development：
1. need to List ignored OAS fields when import OAS3.0 @liuxiran
2. Investigate whether golang has librarys that enables functionality.
cc @juzhiyuan @LiteSun @nic-chen @membphis @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import and Export route
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-730122856
> OpenAPI is a description document used to provide quick reading and testing. The route in APISIX is also used to manage API, there are still some differences. For example, we need to consider how to convert the plugin logic in route.
Yep, OpenApi is mainly defines some of the contents of the api itself, including: `name`, `description`, `service-side address`, `method`, `path`, `request params`, `response params`, `authentication and authorization` , etc, example configurations can refer to [1].
In addition to the basic content of the api mentioned above, `Route` defined in APISIX also contains powerful extension properties, these need to be done by configuring different plugins. The configuration of these plugins are not currently supported by OpenApi except for Authentication plugins.
> We'd better limit the import / export responsibility boundary and provide a more detailed transformation example.
The limit I think would be as follows:
1. Import and export only support the basic information of the api: `name`, `description`, `method`, `path`, `request parameters`, `response parameters`, `authentication`, those are supported by OpenApi, and also the parameters required by APISIX.
1. `Upstream` info, which OpenApi cannot describe very specificly; and other plugins info, which cannot supported by OpenApi, I think we would use the `service module` to achieve this part of functions. As a kind of draft, we can supplement the OpenApi with something like `serviceId field` corresponding to APISIX's service_id[2], before user import their apis, they needs to create a service in APISIX, which can include `upstream` and necessary `plugins`, and add the service_id to their OpenApi configurations.
cc @gxthrj @nic-chen @LiteSun
Reference：
[1]: https://editor.swagger.io/?_ga=2.10529719.2089761848.1605756499-667259753.1599635344
[2]: https://github.com/apache/apisix/blob/master/doc/zh-cn/admin-api.md#route
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran edited a comment on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran edited a comment on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-748009952
After the offline meeting, we have agreed on the following points:
1. First implement import/export based on OAS 3.0, and consider implementing other formats later
2. import and export in atomic.
Todo before development：
1. need to List ignored OAS fields when import OAS3.0 @liuxiran
2. Investigate whether golang has librarys that can support the conversion.
cc @juzhiyuan @LiteSun @nic-chen @membphis @moonming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-dashboard] liuxiran commented on issue #825: [Proposal]: Import & Export route from OpenAPI Specification3.0
Posted by GitBox <gi...@apache.org>.
liuxiran commented on issue #825:
URL: https://github.com/apache/apisix-dashboard/issues/825#issuecomment-749080489
when we import routes from OAS3.0, some fields in OAS will be missed because there are not corresponding fields in APISIX's Route:
1. API General Info: used to describe the general information about your API, some times, a oas file contains a series of apis which belong to a app, so this info is different from the api's name and extra basic info.
[reference](https://swagger.io/docs/specification/api-general-info/)
example:
2. API server and base path: upsream url + url prefix(options)
[reference](https://swagger.io/docs/specification/api-host-and-base-path/)
example:
3. path params: api params described in path
[reference](https://swagger.io/docs/specification/describing-parameters/)
example:
4. query params: api params described in query
[reference](https://swagger.io/docs/specification/describing-parameters/)
example:
5. responses description and links: Define the responses for a API operations
[reference](https://swagger.io/docs/specification/describing-responses/)
example:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org