You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Xiangdong Huang <sa...@gmail.com> on 2021/01/02 06:19:25 UTC
[DISCUSS] Restful API design
Hi,
we'd like to develop some restful APIs for:
1. native Grafana plugin (Grafana uses typescript language, so using
restful API is a solution)
2. other cross-language applications.
According to previous discussions (several months ago on the mailing list),
we will use Netty to implement the REST.
Now, I'd like to raise up a discussion about how to define the API,
especially for authentication. Is someone familiar with it? (Tianci had a
prototype and published it on postman [1]. But I think we still need to do
a lot, e.g., authentication.)
Some interfaces that will be included:
- authentication
- simple write data (write a record, with a device, a timestamp, a number
of measurements)
- simple data query (a number of Paths, raw data query or aggregation
query, time limit, size limit)
- down-sampling query, (a number of Paths, aggregation function,
downsampling time interval, fill function, time limit, size limit)
- any IoTDB SQL.
[1] https://documenter.getpostman.com/view/12320024/T1LLDnuj
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: [DISCUSS] Restful API design
Posted by Xiangdong Huang <sa...@gmail.com>.
Well, if we consider the restful module as a separate process,
then we have to re-implement all the authorization module in the restful
(client sends request -> restful server assign a token to the client, and
cache the privileges of the client.)
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Xiangdong Huang <sa...@gmail.com> 于2021年1月11日周一 上午12:32写道：
> Hi Giorgio,
>
> Thanks, your explanation is detailed.
>
> Currently the restAPI (especially for Grafana query) is designed to
> generate a queryPlan directly without a SQL parser.
>
> If we make the restful service independently, I need to redesign that API.
> (It is ok for me, but just want to hear more voices).
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Giorgio Zoppi <gi...@gmail.com> 于2021年1月10日周日 下午11:13写道：
>
>> Hello,
>> I would apply the single responsibility thing here and create a
>> microservice independent by the main process here. The only burden is to
>> manage
>> the deploy but with docker compose comes easy. The advantages are:
>> - you can put redis in the middle if you want to cache results and lower
>> the latency a lot.
>> - you can benchmark it better.
>> - you can scale the service and the iotdb independently.
>> - one person can only care to update/compile/maintain just the service
>> without thinking too much.
>> - smaller, simple, minimal is good.
>>
>> The discussion is open.
>> Best Regards,
>> Giorgio
>>
>
Re: [DISCUSS] Restful API design
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Giorgio,
Thanks, your explanation is detailed.
Currently the restAPI (especially for Grafana query) is designed to
generate a queryPlan directly without a SQL parser.
If we make the restful service independently, I need to redesign that API.
(It is ok for me, but just want to hear more voices).
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Giorgio Zoppi <gi...@gmail.com> 于2021年1月10日周日 下午11:13写道：
> Hello,
> I would apply the single responsibility thing here and create a
> microservice independent by the main process here. The only burden is to
> manage
> the deploy but with docker compose comes easy. The advantages are:
> - you can put redis in the middle if you want to cache results and lower
> the latency a lot.
> - you can benchmark it better.
> - you can scale the service and the iotdb independently.
> - one person can only care to update/compile/maintain just the service
> without thinking too much.
> - smaller, simple, minimal is good.
>
> The discussion is open.
> Best Regards,
> Giorgio
>
Re: [DISCUSS] Restful API design
Posted by Giorgio Zoppi <gi...@gmail.com>.
Hello,
I would apply the single responsibility thing here and create a
microservice independent by the main process here. The only burden is to
manage
the deploy but with docker compose comes easy. The advantages are:
- you can put redis in the middle if you want to cache results and lower
the latency a lot.
- you can benchmark it better.
- you can scale the service and the iotdb independently.
- one person can only care to update/compile/maintain just the service
without thinking too much.
- smaller, simple, minimal is good.
The discussion is open.
Best Regards,
Giorgio
Re: [DISCUSS] Restful API design
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi, so I'd like to hear suggestions about whether we implement the restful
module as a new independent process or just a module of IoTDB process?
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Giorgio Zoppi <gi...@gmail.com> 于2021年1月10日周日 下午7:49写道：
> Hello,
> as open api is the main format here. I suggest using quarkus
> https://quarkus.io/guides/ or
> https://vertx.io/introduction-to-vertx-and-reactive/
> In our experience Vertx.io scales pretty well.
> https://vertx.io/docs/vertx-web-openapi/java/
>
> https://www.freecodecamp.org/news/an-introduction-to-vert-x-the-fastest-java-framework-today-27d8661ceb14/
>
> Best Regards,
> Giorgio.
>
Re: [DISCUSS] Restful API design
Posted by Giorgio Zoppi <gi...@gmail.com>.
Hello,
as open api is the main format here. I suggest using quarkus
https://quarkus.io/guides/ or
https://vertx.io/introduction-to-vertx-and-reactive/
In our experience Vertx.io scales pretty well.
https://vertx.io/docs/vertx-web-openapi/java/
https://www.freecodecamp.org/news/an-introduction-to-vert-x-the-fastest-java-framework-today-27d8661ceb14/
Best Regards,
Giorgio.
Re: [DISCUSS] Restful API design
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi,
After debugging some source codes of Prometheus and Grafana, I define some
RestFul API using OpenAPI format,
The best thing of using OpenAPI is, we do not need to write to much code
because Swagger or OpenAPI generator can generate the framework.
(Go, Python, Java etc.. Sadly, no native Netty Java codes)
You can get it here [1] or [2].
Thank Yanhong Wang and Tianci Zhu giving the first version of API and
Julian tell me an amazing tool [3] for writing OpenAPI.
[1] https://app.swaggerhub.com/apis/jixuan1989/iotdb/1.0.0
[2] https://issues.apache.org/jira/browse/IOTDB-400
[3] stoplight.io
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Xiangdong Huang <sa...@gmail.com> 于2021年1月2日周六 下午2:19写道：
> Hi,
>
> we'd like to develop some restful APIs for:
>
> 1. native Grafana plugin (Grafana uses typescript language, so using
> restful API is a solution)
>
> 2. other cross-language applications.
>
> According to previous discussions (several months ago on the mailing
> list), we will use Netty to implement the REST.
>
> Now, I'd like to raise up a discussion about how to define the API,
> especially for authentication. Is someone familiar with it? (Tianci had a
> prototype and published it on postman [1]. But I think we still need to do
> a lot, e.g., authentication.)
>
> Some interfaces that will be included:
> - authentication
> - simple write data (write a record, with a device, a timestamp, a
> number of measurements)
> - simple data query (a number of Paths, raw data query or aggregation
> query, time limit, size limit)
> - down-sampling query, (a number of Paths, aggregation function,
> downsampling time interval, fill function, time limit, size limit)
> - any IoTDB SQL.
>
> [1] https://documenter.getpostman.com/view/12320024/T1LLDnuj
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>