答复: [External] Re: Commit tsfile-go

[Lyndon Dong5 Li <li...@lenovo.com>]

Hi,

After investigation, I found the tsfile format of go version is almost the same with the latest TsFile format of java version, but unfortunately there are some differences between them: They have almost the same meta objects structures, but there are several attributes are not exactly the same.

Maybe we can eliminate these differences together if necessary.


-----邮件原件-----
发件人: Xiangdong Huang <sa...@gmail.com> 
发送时间: 2019年4月1日 23:03
收件人: dev@iotdb.apache.org
主题: [External] Re: Commit tsfile-go

Hi,

Glad to see that Lenovo makes this module open sourced.

I want to confirm that Is this go version follow the latest TsFile format?
see https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format

Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Lyndon Dong5 Li <li...@lenovo.com> 于2019年4月1日周一 下午7:33写道：

> Hi,
>
> I'm Lidong from Lenovo, I've just created a pull request to commit 
> project 'tsfile-go' to branch 'master'.
> Tsfile-go is a Golang version of tsfile(based on branch 'thanos'), 
> which was developed by Lenovo & TsingHua. we implemented all of 
> features in 'tsfile', including:
> 1. Writing ts data to a tsfile
> 2. Reading/querying ts data from an existing file 3. Encoding/decoding 
> with RLE/TS_2DIFF/GORILLA/PLAIN 4. Compression/decompression with 
> snappy
>
> Now we have applied tsfile-go to Lenovo IOT platform, for purpose of 
> caching & compressing ts data on edge devices.
>
> Best regards
> -----------------------------------
> Lidong
> LCIG Big Data BU, Lenovo
>
>

Re: [External] Re: Commit tsfile-go

[江天 <jt...@163.com>]

Yes, I think the main differences will be: 
1. The header of a ChunkGroup(called RowGroup previously) is moved to the tail of the data and becomes a footer.
2. As a consequence of 1., there is a marker before each ChunkHeader and ChunkGroupFooter, and there is also a marker before the index at the tail of the file, indicating that the data section is over. 
3. The index structure at the tail of a TsFile also changes.
Please point it out if I miss anything.

> 在 2019年4月2日，下午5:10，Lyndon Dong5 Li <li...@lenovo.com> 写道：
> 
> Hi,
> 
> After investigation, I found the tsfile format of go version is almost the same with the latest TsFile format of java version, but unfortunately there are some differences between them: They have almost the same meta objects structures, but there are several attributes are not exactly the same.
> 
> Maybe we can eliminate these differences together if necessary.
> 
> 
> -----邮件原件-----
> 发件人: Xiangdong Huang <sa...@gmail.com> 
> 发送时间: 2019年4月1日 23:03
> 收件人: dev@iotdb.apache.org
> 主题: [External] Re: Commit tsfile-go
> 
> Hi,
> 
> Glad to see that Lenovo makes this module open sourced.
> 
> I want to confirm that Is this go version follow the latest TsFile format?
> see https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format
> 
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> Lyndon Dong5 Li <li...@lenovo.com> 于2019年4月1日周一 下午7:33写道：
> 
>> Hi,
>> 
>> I'm Lidong from Lenovo, I've just created a pull request to commit 
>> project 'tsfile-go' to branch 'master'.
>> Tsfile-go is a Golang version of tsfile(based on branch 'thanos'), 
>> which was developed by Lenovo & TsingHua. we implemented all of 
>> features in 'tsfile', including:
>> 1. Writing ts data to a tsfile
>> 2. Reading/querying ts data from an existing file 3. Encoding/decoding 
>> with RLE/TS_2DIFF/GORILLA/PLAIN 4. Compression/decompression with 
>> snappy
>> 
>> Now we have applied tsfile-go to Lenovo IOT platform, for purpose of 
>> caching & compressing ts data on edge devices.
>> 
>> Best regards
>> -----------------------------------
>> Lidong
>> LCIG Big Data BU, Lenovo
>> 
>>