You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Xiangdong Huang <sa...@gmail.com> on 2020/10/10 07:40:41 UTC
Does someone want to maintain the TsFile Golang version
Hi,
Last year... someone (as far as I know, from Lenovo) proposed a Tsfile
read/write API implementation with GoLang[1].
The file format is consistent with v0.8.0, but we have upgraded to v0.11
(the file format is the same with 0.10, version 000002).
Does someone want to maintain this PR?
The new TsFile format is here [2].
[1] https://github.com/apache/iotdb/pull/124
[2] https://iotdb.apache.org/SystemDesign/TsFile/Format.html
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: Does someone want to maintain the TsFile Golang version
Posted by Giorgio Zoppi <gi...@gmail.com>.
Hi Xiangdong,
I was looking for cutting work instead of creating more work. It is true
that it's no zero cost abstraction.
The question is here: Does it make sense having a well-known serialization
format for avoiding *to write a parser*?
Here is the list:
https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formats
Is there any of those format, that will fit to TsFile ideas and design?
If not, i am fine, i will help with Go and C++.
Best Regards,
Giorgio.
Re: Does someone want to maintain the TsFile Golang version
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Giorgio,
If just IDL (Interface description language), then we still implement these
interfaces using different languages.
I do not think protobuf works. It can describe the format of TsFile, but
using protobuf to serialize data maybe not a good idea,
as we then are hard to know the meaning of each byte.
Besides, I am not sure protobuf can provide the best compression.
e.g., the first 12 bytes are magic string + version. If using protobuf, how
to constrain that?
Maybe many fields will have to be defined as bytes (define them as string
will waste space as we know the length of the string).
But maybe we can do some experiments.
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Giorgio Zoppi <gi...@gmail.com> 于2020年10月11日周日 下午4:19写道：
> Hello,
> We definitely need an IDL for the TsFile, from the IDL we can define use a
> serialization mechanism that is not language dependent.
> Can TsFile be described with protobuf for example?
> BR,
> Giorgio
>
Re: Does someone want to maintain the TsFile Golang version
Posted by Giorgio Zoppi <gi...@gmail.com>.
Hello,
We definitely need an IDL for the TsFile, from the IDL we can define use a
serialization mechanism that is not language dependent.
Can TsFile be described with protobuf for example?
BR,
Giorgio
Re: Does someone want to maintain the TsFile Golang version
Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Hi Chris,
Recently we found that there is a lot room for improvement in the TsFile(version 000002) [1].
We could design the new TsFile structure (version 000003) together and then consider the go version.
[1] http://iotdb.apache.org/SystemDesign/TsFile/Format.html
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "Xiangdong Huang" <sa...@gmail.com>
> 发送时间: 2020-10-10 15:55:46 (星期六)
> 收件人: "Christofer Dutz" <ch...@c-ware.de>
> 抄送: dev <de...@iotdb.apache.org>
> 主题: Re: Does someone want to maintain the TsFile Golang version
>
> Hi Chris,
>
> Nice!
>
> > But as I started learning go on Tuesday, I wouldn't call myself s senor
> go developer.
>
> More senior than me. :D
>
> By the way, tools/tsfileToolSet/print-tsfile-sketch.sh will provide help to
> understand the format of TsFile.
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Christofer Dutz <ch...@c-ware.de> 于2020年10月10日周六 下午3:49写道：
>
> > As I'm currently working on plc4go I could possibly join in :-)
> >
> > But as I started learning go on Tuesday, I wouldn't call myself s senor go
> > developer.
> >
> > Chris
> >
> > ------------------------------
> > *Von:* Xiangdong Huang <sa...@gmail.com>
> > *Gesendet:* Samstag, 10. Oktober 2020 09:40
> > *An:* dev <de...@iotdb.apache.org>
> > *Betreff:* Does someone want to maintain the TsFile Golang version
> >
> > Hi,
> >
> > Last year... someone (as far as I know, from Lenovo) proposed a Tsfile
> > read/write API implementation with GoLang[1].
> >
> > The file format is consistent with v0.8.0, but we have upgraded to v0.11
> > (the file format is the same with 0.10, version 000002).
> >
> > Does someone want to maintain this PR?
> > The new TsFile format is here [2].
> >
> > [1] https://github.com/apache/iotdb/pull/124
> > [2] https://iotdb.apache.org/SystemDesign/TsFile/Format.html
> >
> > Best,
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
Re: Does someone want to maintain the TsFile Golang version
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Chris,
Nice!
> But as I started learning go on Tuesday, I wouldn't call myself s senor
go developer.
More senior than me. :D
By the way, tools/tsfileToolSet/print-tsfile-sketch.sh will provide help to
understand the format of TsFile.
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Christofer Dutz <ch...@c-ware.de> 于2020年10月10日周六 下午3:49写道：
> As I'm currently working on plc4go I could possibly join in :-)
>
> But as I started learning go on Tuesday, I wouldn't call myself s senor go
> developer.
>
> Chris
>
> ------------------------------
> *Von:* Xiangdong Huang <sa...@gmail.com>
> *Gesendet:* Samstag, 10. Oktober 2020 09:40
> *An:* dev <de...@iotdb.apache.org>
> *Betreff:* Does someone want to maintain the TsFile Golang version
>
> Hi,
>
> Last year... someone (as far as I know, from Lenovo) proposed a Tsfile
> read/write API implementation with GoLang[1].
>
> The file format is consistent with v0.8.0, but we have upgraded to v0.11
> (the file format is the same with 0.10, version 000002).
>
> Does someone want to maintain this PR?
> The new TsFile format is here [2].
>
> [1] https://github.com/apache/iotdb/pull/124
> [2] https://iotdb.apache.org/SystemDesign/TsFile/Format.html
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
Re: Does someone want to maintain the TsFile Golang version
Posted by Christofer Dutz <ch...@c-ware.de>.
As I'm currently working on plc4go I could possibly join in :-)
But as I started learning go on Tuesday, I wouldn't call myself s senor go developer.
Chris
________________________________
Von: Xiangdong Huang <sa...@gmail.com>
Gesendet: Samstag, 10. Oktober 2020 09:40
An: dev <de...@iotdb.apache.org>
Betreff: Does someone want to maintain the TsFile Golang version
Hi,
Last year... someone (as far as I know, from Lenovo) proposed a Tsfile
read/write API implementation with GoLang[1].
The file format is consistent with v0.8.0, but we have upgraded to v0.11
(the file format is the same with 0.10, version 000002).
Does someone want to maintain this PR?
The new TsFile format is here [2].
[1] https://github.com/apache/iotdb/pull/124
[2] https://iotdb.apache.org/SystemDesign/TsFile/Format.html
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院