You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Julian Feinauer <j....@pragmaticminds.de> on 2021/09/28 10:33:21 UTC

IOTDB-Lite as sqllite approach

Hi folks,

I just came up with a little idea to improve the usage of TSFiles at the „edge“. What if we create an access layer (besides the Readers /Writers) to access the TSFiles with SQL as if they were in the server.
Thats basically the same that sqlite does with extreme success (https://sqlite.org/index.html).

For Java I created a very minimal sketch based on Apache Calcite (so with a different SQL Dialect than the server but this could be adapted), here: https://github.com/JulianFeinauer/incubator-iotdb/tree/feature/iotdb-lite/iotdb-lite

What do you think about that?

Happy to hear many ideas and feedback : )

Julian

AW: IOTDB-Lite as sqllite approach

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi Giorgio and Xiangdong,

I agree with both of you.
And as initially indicated I think the long term perfect way would be to have a C based implementation for the whole "process" with wrappers for many languages and ecosystems (as SQLITE does it).

And I agree that the advantage in a Java setup is not much although I sometimes see the advantage of having something running in the main process of an application instead of having a separate service (Android is a good example, indeed).

Best
Julian

________________________________
Von: Giorgio Zoppi <gi...@gmail.com>
Gesendet: Mittwoch, 29. September 2021 03:20
An: dev <de...@iotdb.apache.org>
Betreff: Re: IOTDB-Lite as sqllite approach

We might want to support TsFile in different languages: Go, Rust, C and C++
as parquet/arrow people are doing.

Re: IOTDB-Lite as sqllite approach

Posted by Giorgio Zoppi <gi...@gmail.com>.
We might want to support TsFile in different languages: Go, Rust, C and C++
as parquet/arrow people are doing.

Re: IOTDB-Lite as sqllite approach

Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Julian,

Let TsFile supporting SQL is attractive.

However, as it may have some limitations, e.g.,  a tsfile does not
support out-of-order data integration for a given time series.
It will limit the usage scenarios: if a embedded machine can run
TsFile with Java, why not run IoTDB ....

The above question may have two reasons:
 - computation resource limitation, but I think if the workload is not
heavy, IoTDB will also spend few.
 - The OS/application does not allow running a long-term process,
i.e., an embedded database is needed, and that is what SQLLite
suitable for.

For reason one, I think TsFile (C) is the final solution.

For reason two, I think TsFile (Java) + SQL layer is very suitable.
For example, considering it runs on an Android OS based machine.


Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院

Julian Feinauer <j....@pragmaticminds.de> 于2021年9月28日周二 下午6:33写道:
>
> Hi folks,
>
> I just came up with a little idea to improve the usage of TSFiles at the „edge“. What if we create an access layer (besides the Readers /Writers) to access the TSFiles with SQL as if they were in the server.
> Thats basically the same that sqlite does with extreme success (https://sqlite.org/index.html).
>
> For Java I created a very minimal sketch based on Apache Calcite (so with a different SQL Dialect than the server but this could be adapted), here: https://github.com/JulianFeinauer/incubator-iotdb/tree/feature/iotdb-lite/iotdb-lite
>
> What do you think about that?
>
> Happy to hear many ideas and feedback : )
>
> Julian