You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by 刘威 <ti...@qq.com.INVALID> on 2021/08/04 04:54:28 UTC

Progress Report for IoTDB-Skywalking Adapter

Hi, I'm a participant in Summer 2021 of Open Source Promotion Plan. My name is Liu Wei. 
My project is create a IoTDB-Adapter for Skywalking. (Project Link: https://summer.iscas.ac.cn/#/org/prodetail/210070771)

I have pushed my progress to my Github fork (https://github.com/LIU-WEI-git/skywalking) and official GitLab (https://gitlab.summer-ospp.ac.cn/summer2021/210070771)
My current solution is similar to relational database. It is introduced in README.md on https://github.com/LIU-WEI-git/skywalking/tree/iotdb-adapter/oap-server/server-storage-plugin/storage-iotdb-plugin. What's more, I also introduce another solution with index in README.md and will adopt it later.

I have implement all insert, update, delete method and part of query method. I will push them together when I finish the rest.
If you have any suggestion, please give it to me, thanks in advance. 

I have found much different query size limit in the other storage plugins(in application.yml). Dose it relate to upper applications?
How many default pieces of data should be queried from those model, such as Segment, ProfileTask, ProfileTaskLog, ProfileThreadSnapshot, NetworkAddressAlias and MetaData? 
Should I set the same query size limit for IoTDB storage plugin?

Re: Progress Report for IoTDB-Skywalking Adapter

Posted by Sheng Wu <wu...@gmail.com>.
If you have questions(I saw those in the readme), you could set up an
online meeting, I am glad to help.

Sheng Wu 吴晟
Twitter, wusheng1108

Sheng Wu <wu...@gmail.com> 于2021年8月4日周三 下午2:06写道：
>
> Hi
>
> I can see you provided an unindexed solution, which could be used as a
> POC implementation.
> If you want end users really to adopt this, we need to make sure of
> the performance.
> Metric is a million-level dataset as the number of service instances
> and endpoints.
> Log and Trace are a billion level dataset
>
> N K write/s op for the metrics.
> The 10 * number of cluster services * traffic is the scale of trace
> and log, even more.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
> 刘威 <ti...@qq.com.invalid> 于2021年8月4日周三 下午12:54写道：
> >
> > Hi, I'm a participant in Summer 2021 of Open Source Promotion Plan. My name is Liu Wei.
> > My project is create a IoTDB-Adapter for Skywalking. (Project Link: https://summer.iscas.ac.cn/#/org/prodetail/210070771)
> >
> > I have pushed my progress to my Github fork (https://github.com/LIU-WEI-git/skywalking) and official GitLab (https://gitlab.summer-ospp.ac.cn/summer2021/210070771)
> > My current solution is similar to relational database. It is introduced in README.md on https://github.com/LIU-WEI-git/skywalking/tree/iotdb-adapter/oap-server/server-storage-plugin/storage-iotdb-plugin. What's more, I also introduce another solution with index in README.md and will adopt it later.
> >
> > I have implement all insert, update, delete method and part of query method. I will push them together when I finish the rest.
> > If you have any suggestion, please give it to me, thanks in advance.
> >
> > I have found much different query size limit in the other storage plugins(in application.yml). Dose it relate to upper applications?
> > How many default pieces of data should be queried from those model, such as Segment, ProfileTask, ProfileTaskLog, ProfileThreadSnapshot, NetworkAddressAlias and MetaData?
> > Should I set the same query size limit for IoTDB storage plugin?

Re: Progress Report for IoTDB-Skywalking Adapter

Posted by Sheng Wu <wu...@gmail.com>.
If you have questions(I saw those in the readme), you could set up an
online meeting, I am glad to help.

Sheng Wu 吴晟
Twitter, wusheng1108

Sheng Wu <wu...@gmail.com> 于2021年8月4日周三 下午2:06写道：
>
> Hi
>
> I can see you provided an unindexed solution, which could be used as a
> POC implementation.
> If you want end users really to adopt this, we need to make sure of
> the performance.
> Metric is a million-level dataset as the number of service instances
> and endpoints.
> Log and Trace are a billion level dataset
>
> N K write/s op for the metrics.
> The 10 * number of cluster services * traffic is the scale of trace
> and log, even more.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
> 刘威 <ti...@qq.com.invalid> 于2021年8月4日周三 下午12:54写道：
> >
> > Hi, I'm a participant in Summer 2021 of Open Source Promotion Plan. My name is Liu Wei.
> > My project is create a IoTDB-Adapter for Skywalking. (Project Link: https://summer.iscas.ac.cn/#/org/prodetail/210070771)
> >
> > I have pushed my progress to my Github fork (https://github.com/LIU-WEI-git/skywalking) and official GitLab (https://gitlab.summer-ospp.ac.cn/summer2021/210070771)
> > My current solution is similar to relational database. It is introduced in README.md on https://github.com/LIU-WEI-git/skywalking/tree/iotdb-adapter/oap-server/server-storage-plugin/storage-iotdb-plugin. What's more, I also introduce another solution with index in README.md and will adopt it later.
> >
> > I have implement all insert, update, delete method and part of query method. I will push them together when I finish the rest.
> > If you have any suggestion, please give it to me, thanks in advance.
> >
> > I have found much different query size limit in the other storage plugins(in application.yml). Dose it relate to upper applications?
> > How many default pieces of data should be queried from those model, such as Segment, ProfileTask, ProfileTaskLog, ProfileThreadSnapshot, NetworkAddressAlias and MetaData?
> > Should I set the same query size limit for IoTDB storage plugin?

Re: Progress Report for IoTDB-Skywalking Adapter

Posted by Sheng Wu <wu...@gmail.com>.
Hi

I can see you provided an unindexed solution, which could be used as a
POC implementation.
If you want end users really to adopt this, we need to make sure of
the performance.
Metric is a million-level dataset as the number of service instances
and endpoints.
Log and Trace are a billion level dataset

N K write/s op for the metrics.
The 10 * number of cluster services * traffic is the scale of trace
and log, even more.

Sheng Wu 吴晟
Twitter, wusheng1108

刘威 <ti...@qq.com.invalid> 于2021年8月4日周三 下午12:54写道：
>
> Hi, I'm a participant in Summer 2021 of Open Source Promotion Plan. My name is Liu Wei.
> My project is create a IoTDB-Adapter for Skywalking. (Project Link: https://summer.iscas.ac.cn/#/org/prodetail/210070771)
>
> I have pushed my progress to my Github fork (https://github.com/LIU-WEI-git/skywalking) and official GitLab (https://gitlab.summer-ospp.ac.cn/summer2021/210070771)
> My current solution is similar to relational database. It is introduced in README.md on https://github.com/LIU-WEI-git/skywalking/tree/iotdb-adapter/oap-server/server-storage-plugin/storage-iotdb-plugin. What's more, I also introduce another solution with index in README.md and will adopt it later.
>
> I have implement all insert, update, delete method and part of query method. I will push them together when I finish the rest.
> If you have any suggestion, please give it to me, thanks in advance.
>
> I have found much different query size limit in the other storage plugins(in application.yml). Dose it relate to upper applications?
> How many default pieces of data should be queried from those model, such as Segment, ProfileTask, ProfileTaskLog, ProfileThreadSnapshot, NetworkAddressAlias and MetaData?
> Should I set the same query size limit for IoTDB storage plugin?

Re: Progress Report for IoTDB-Skywalking Adapter

Posted by Sheng Wu <wu...@gmail.com>.
Hi

I can see you provided an unindexed solution, which could be used as a
POC implementation.
If you want end users really to adopt this, we need to make sure of
the performance.
Metric is a million-level dataset as the number of service instances
and endpoints.
Log and Trace are a billion level dataset

N K write/s op for the metrics.
The 10 * number of cluster services * traffic is the scale of trace
and log, even more.

Sheng Wu 吴晟
Twitter, wusheng1108

刘威 <ti...@qq.com.invalid> 于2021年8月4日周三 下午12:54写道：
>
> Hi, I'm a participant in Summer 2021 of Open Source Promotion Plan. My name is Liu Wei.
> My project is create a IoTDB-Adapter for Skywalking. (Project Link: https://summer.iscas.ac.cn/#/org/prodetail/210070771)
>
> I have pushed my progress to my Github fork (https://github.com/LIU-WEI-git/skywalking) and official GitLab (https://gitlab.summer-ospp.ac.cn/summer2021/210070771)
> My current solution is similar to relational database. It is introduced in README.md on https://github.com/LIU-WEI-git/skywalking/tree/iotdb-adapter/oap-server/server-storage-plugin/storage-iotdb-plugin. What's more, I also introduce another solution with index in README.md and will adopt it later.
>
> I have implement all insert, update, delete method and part of query method. I will push them together when I finish the rest.
> If you have any suggestion, please give it to me, thanks in advance.
>
> I have found much different query size limit in the other storage plugins(in application.yml). Dose it relate to upper applications?
> How many default pieces of data should be queried from those model, such as Segment, ProfileTask, ProfileTaskLog, ProfileThreadSnapshot, NetworkAddressAlias and MetaData?
> Should I set the same query size limit for IoTDB storage plugin?