TsFile Go use case

[HuiA Dai <da...@lenovo.com>]

Dear all,

I am Daihui, and I submitted a PR [1] which uses Go to implement TsFile.

I want to share with all of you that our company (Lenovo) have used this feature in our business case in YOFC (Yangtze Optical Fiber and Cable Company) as the time series data storage of the edge computing in the energy management system. The system collects data from more than 90 thousand data points, stores them to the TsFile (with Go) as the edge storage and then reads them later to send to the cloud. Until now the system had run stably for more than 1 year.

The TsFile format of the Go version is inconsistent with the format of current Java version, because TsFile format has upgraded from v0.8 to v0.9.

Later we may upgrade it and share it to the community. We welcome who are interested in this feature contribute to the feature together.

[1] https://github.com/apache/incubator-iotdb/pull/124

Best,

DAI HUI

Re: TsFile Go use case

[Giorgio Zoppi <gi...@gmail.com>]

Dear all,
it will be interesting knowing more on how we can simulate (have a test)
your Lenovo use case so we can make performance comparison.
Best Regards,
Giorgio.

Re: TsFile Go use case

[Xiangdong Huang <sa...@gmail.com>]

Hi,

Thanks for sharing the use case.

It is quite similar to the issue about IoTDB-Native while the issue is for
implementing TsFile with C++ currently.

We may have several implementations of TsFile, by Java, Go, C++... Then, it
is easier to use TsFile in many applications/environments (and maybe with
better performance and less resources).

Look forward to your contributions.

Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Dawei Liu <at...@163.com> 于2020年5月9日周六 下午7:26写道：

> Hi Daihui,
>
>
> Thank you very much for sharing your use of iotdb in the mail.
> I have noticed your pr for a long time,
> and I think it is necessary to discuss whether to merge the golang code
> into iotdb github,
> So that we may have TsFile for C++ API, golang API and Java API in the
> future.
> TsFile can be maintained separately as a big data ecosystem, and can
> cooperate with hudi, spark, flink, etc.
>
>
>
>
>
>
> Thanks
> ---
> Dawei Liu
>
>
>
> On 05/9/2020 15:15，HuiA Dai<da...@lenovo.com> wrote：
> Dear all,
>
> I am Daihui, and I submitted a PR [1] which uses Go to implement TsFile.
>
> I want to share with all of you that our company (Lenovo) have used this
> feature in our business case in YOFC (Yangtze Optical Fiber and Cable
> Company) as the time series data storage of the edge computing in the
> energy management system. The system collects data from more than 90
> thousand data points, stores them to the TsFile (with Go) as the edge
> storage and then reads them later to send to the cloud. Until now the
> system had run stably for more than 1 year.
>
> The TsFile format of the Go version is inconsistent with the format of
> current Java version, because TsFile format has upgraded from v0.8 to v0.9.
>
> Later we may upgrade it and share it to the community. We welcome who are
> interested in this feature contribute to the feature together.
>
> [1] https://github.com/apache/incubator-iotdb/pull/124
>
> Best,
>
> DAI HUI
>
>

Re:TsFile Go use case

[Dawei Liu <at...@163.com>]

Hi Daihui,


Thank you very much for sharing your use of iotdb in the mail. 
I have noticed your pr for a long time, 
and I think it is necessary to discuss whether to merge the golang code into iotdb github, 
So that we may have TsFile for C++ API, golang API and Java API in the future.
TsFile can be maintained separately as a big data ecosystem, and can cooperate with hudi, spark, flink, etc.






Thanks
---
Dawei Liu



On 05/9/2020 15:15，HuiA Dai<da...@lenovo.com> wrote：
Dear all,

I am Daihui, and I submitted a PR [1] which uses Go to implement TsFile.

I want to share with all of you that our company (Lenovo) have used this feature in our business case in YOFC (Yangtze Optical Fiber and Cable Company) as the time series data storage of the edge computing in the energy management system. The system collects data from more than 90 thousand data points, stores them to the TsFile (with Go) as the edge storage and then reads them later to send to the cloud. Until now the system had run stably for more than 1 year.

The TsFile format of the Go version is inconsistent with the format of current Java version, because TsFile format has upgraded from v0.8 to v0.9.

Later we may upgrade it and share it to the community. We welcome who are interested in this feature contribute to the feature together.

[1] https://github.com/apache/incubator-iotdb/pull/124

Best,

DAI HUI