You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Dhruv Garg <dg...@gmail.com> on 2021/03/02 07:53:56 UTC
New user of IoTDB - a few queries
Hello,
I am a researcher and a new user of IoTDB. I was excited when I saw the
performance numbers of IoTDB compared to InfluxDB, and I have set it up on
a few workstations and a Raspberry Pi device.
I am becoming familiar with it and have a few queries:
1. Does IoTDB support Join queries? - I tried looking up some
documentation on it, but could not find any. If the feature is available,
could you please provide a sample query or the predicates for join queries?
2. I also browsed through the IoTDB Jira issues page. I saw that the
distributed / cluster version is being developed. Approximately how much
time would it take to get a stable / working version of the cluster IoTDB?
It would be great if someone can respond to my queries. I would be
waiting for a reply. Thanks in advance.
Regards,
dgargcs
Re: Re: New user of IoTDB - a few queries
Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Hi Dhruv,
Welcome! Here is some design doc[1] about the query process :)
[1] http://iotdb.apache.org/SystemDesign/DataQuery/DataQuery.html
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "Dhruv Garg" <dg...@gmail.com>
> 发送时间: 2021-03-08 21:51:21 (星期一)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: Re: New user of IoTDB - a few queries
>
> Yep, thanks! I would like to. Just need to find the time. :P
>
> Regards,
> dgargcs
>
> On Mon, 8 Mar 2021 at 17:40, Xiangdong Huang <sa...@gmail.com> wrote:
>
> > By the way, welcome to contribute left, right, inner join to IoTDB if you
> > like ~ :D
> >
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
> >
> > Xiangdong Huang <sa...@gmail.com> 于2021年3月8日周一 下午8:09写道：
> >
> > > Hi, yes, root is like "/" in Unix-like OS.
> > >
> > > -----------------------------------
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > > 黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午6:14写道：
> > >
> > >> Hello Xiangdong,
> > >>
> > >> Thanks for confirming! One last question for now, all storage groups
> > must
> > >> start with "root" right? I cannot have storage groups like root1.demo,
> > >> correct?
> > >>
> > >> Regards,
> > >> dgargcs
> > >>
> > >> On Mon, 8 Mar 2021 at 15:37, Xiangdong Huang <sa...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Dhruv,
> > >> >
> > >> > Yes you are right. and IoTDB also supports join on timestamp across
> > >> > storage groups.
> > >> >
> > >> > Q2: IoTDB does not support that now.
> > >> > Current join looks like outer-join in RDBMS.
> > >> > Supporting left join, right join and inner-join is not hard to
> > >> implement,
> > >> > just omit the rows that contain NULL.
> > >> > (Better implementation will need CBO cost-based optimization, but
> > IoTDB
> > >> > lacks the statistics info for that.)
> > >> >
> > >> > Best,
> > >> > -----------------------------------
> > >> > Xiangdong Huang
> > >> > School of Software, Tsinghua University
> > >> >
> > >> > 黄向东
> > >> > 清华大学 软件学院
> > >> >
> > >> >
> > >> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
> > >> >
> > >> > > Hello Xiangdong,
> > >> > >
> > >> > > Yes. Let me elaborate a bit about the join requirement. Suppose I
> > >> want to
> > >> > > do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2
> > >> for
> > >> > a
> > >> > > common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It
> > >> > should
> > >> > > print out the last two rows from your table.
> > >> > >
> > >> > > In SQL, when we do a join, there are two different tables and then
> > we
> > >> do
> > >> > a
> > >> > > join based on some common column => root.demo.d1 is one table (has
> > an
> > >> > "m1"
> > >> > > column) and another table root.demo.d2 (has an "m2" column). If I
> > do a
> > >> > join
> > >> > > on d1 and d2 based on timestamp, it will return some rows containing
> > >> m1
> > >> > and
> > >> > > m2 where timestamps are the same.
> > >> > >
> > >> > > I think the way to look at join queries in IoTDB is slightly
> > >> different.
> > >> > > Here, we have a storage group (root.demo) with different time-series
> > >> > (d1.m1
> > >> > > and d2.m2). For each insert in a storage group, I think IoTDB
> > >> > automatically
> > >> > > joins these time-series on timestamp since it assumes null values
> > for
> > >> > other
> > >> > > time-series that were not given a value during insert. For example,
> > >> if I
> > >> > > do an "insert root.demo.d2(timestamp,m2) values
> > (2021-08-03T00:00:00,
> > >> > > 2021)", it will also append NULL value for root.demo.d1.m1 for this
> > >> > > timestamp.
> > >> > >
> > >> > > Thus, to do a join within a storage group for *any number* of
> > >> time-series
> > >> > > is straightforward. I think this is trivially done by just giving
> > the
> > >> > time
> > >> > > range and time-series names => "select d1.m1, d2.m2 from root.demo
> > >> where
> > >> > > timestamp >= 2019-19-11T00:00:00".
> > >> > >
> > >> > > Next, I thought about a similar query across storage groups, say
> > >> > root.demo1
> > >> > > and root.demo2. Here as well, I can run a similar query, but only
> > >> start
> > >> > > from a higher level in the storage hierarchy, i.e. "root" alone.
> > >> "select
> > >> > > demo1.m1, demo2.m2 from root where timestamp >=
> > 2019-19-11T00:00:00".
> > >> > >
> > >> > > Q1. Is my interpretation of join on timestamp correct for IoTDB?
> > >> > >
> > >> > > Q2. Is there a way to omit rows that return one or more null column
> > >> > values
> > >> > > in the result?
> > >> > >
> > >> > > PS: Sorry for the long email. I am exploring IoTDB for a research
> > >> project
> > >> > > and just want to be clear in my understanding.
> > >> > >
> > >> > > Thanks in advance!
> > >> > >
> > >> > > Regards,
> > >> > > dgargcs
> > >> > >
> > >> > > On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com>
> > >> wrote:
> > >> > >
> > >> > > > Hi Dhruv,
> > >> > > >
> > >> > > > 1. Could you give me an example to describe what you want to
> > query?
> > >> > > > e.g., there are two time series:
> > >> > > >
> > >> > > > +-----------------------------+---------------+---------------+
> > >> > > > | Time|root.demo.d1.m1|root.demo.d2.m2|
> > >> > > > +-----------------------------+---------------+---------------+
> > >> > > > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> > >> > > > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> > >> > > > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> > >> > > > +-----------------------------+---------------+---------------+
> > >> > > >
> > >> > > > or,
> > >> > > >
> > >> > > > What you want to query? (you can consider it as a table in MySQL,
> > >> and
> > >> > > just
> > >> > > > write the sqls you need in mysql.)
> > >> > > >
> > >> > > > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I
> > >> > think.
> > >> > > >
> > >> > > > Best,
> > >> > > > -----------------------------------
> > >> > > > Xiangdong Huang
> > >> > > > School of Software, Tsinghua University
> > >> > > >
> > >> > > > 黄向东
> > >> > > > 清华大学 软件学院
> > >> > > >
> > >> > > >
> > >> > > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> > >> > > >
> > >> > > > > Hello Xiangdong,
> > >> > > > >
> > >> > > > > Thanks for your response.
> > >> > > > >
> > >> > > > > 1. I have the same use-case: I want to run a join query on two
> > >> time
> > >> > > > > series but I think I am getting the syntax wrong.
> > >> > > > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> > >> > > > following
> > >> > > > > error: "mismatched input 'JOIN' expecting {<eof>, WHERE, ORDER,
> > >> > > SLIMIT,
> > >> > > > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> > >> > > > > Could you please refer me to some documentation or provide a
> > >> simple
> > >> > > > example
> > >> > > > > join query for IoTDB?
> > >> > > > >
> > >> > > > > 2. I am also excited to try out the cluster version. Is there an
> > >> > > > > approximate timeline for v0.12 rollout?
> > >> > > > >
> > >> > > > > Thanks in advance.
> > >> > > > >
> > >> > > > > Regards,
> > >> > > > > dgargcs
> > >> > > > >
> > >> > > > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <
> > sainthxd@gmail.com>
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > Hi Dhruv,
> > >> > > > > >
> > >> > > > > > Welcome!
> > >> > > > > >
> > >> > > > > > 1. "join" for what requirements?
> > >> > > > > >
> > >> > > > > > IoTDB supports JOIN two time series ON their timestamp, i.e.,
> > >> time
> > >> > > > series
> > >> > > > > > alignment.
> > >> > > > > >
> > >> > > > > > 2. I think the cluster version is coming... In the coming
> > >> v0.12, I
> > >> > > > think
> > >> > > > > a
> > >> > > > > > cluster version (that does
> > >> > > > > > not support dynamically scale-out) will be released, and the
> > >> > > scale-out
> > >> > > > > > feature may be released in v0.13 or 0.14.
> > >> > > > > >
> > >> > > > > > Best,
> > >> > > > > > -----------------------------------
> > >> > > > > > Xiangdong Huang
> > >> > > > > > School of Software, Tsinghua University
> > >> > > > > >
> > >> > > > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> > >> > > > > >
> > >> > > > > > > Hello,
> > >> > > > > > >
> > >> > > > > > > I am a researcher and a new user of IoTDB. I was excited
> > when
> > >> I
> > >> > saw
> > >> > > > the
> > >> > > > > > > performance numbers of IoTDB compared to InfluxDB, and I
> > have
> > >> set
> > >> > > it
> > >> > > > up
> > >> > > > > > on
> > >> > > > > > > a few workstations and a Raspberry Pi device.
> > >> > > > > > >
> > >> > > > > > > I am becoming familiar with it and have a few queries:
> > >> > > > > > >
> > >> > > > > > > 1. Does IoTDB support Join queries? - I tried looking up
> > >> some
> > >> > > > > > > documentation on it, but could not find any. If the
> > >> feature is
> > >> > > > > > > available,
> > >> > > > > > > could you please provide a sample query or the predicates
> > >> for
> > >> > > join
> > >> > > > > > > queries?
> > >> > > > > > > 2. I also browsed through the IoTDB Jira issues page. I
> > saw
> > >> > that
> > >> > > > the
> > >> > > > > > > distributed / cluster version is being developed.
> > >> > Approximately
> > >> > > > how
> > >> > > > > > much
> > >> > > > > > > time would it take to get a stable / working version of
> > the
> > >> > > > cluster
> > >> > > > > > > IoTDB?
> > >> > > > > > >
> > >> > > > > > > It would be great if someone can respond to my queries. I
> > >> would
> > >> > be
> > >> > > > > > > waiting for a reply. Thanks in advance.
> > >> > > > > > >
> > >> > > > > > > Regards,
> > >> > > > > > > dgargcs
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
</d...@gmail.com>
Re: New user of IoTDB - a few queries
Posted by Dhruv Garg <dg...@gmail.com>.
Yep, thanks! I would like to. Just need to find the time. :P
Regards,
dgargcs
On Mon, 8 Mar 2021 at 17:40, Xiangdong Huang <sa...@gmail.com> wrote:
> By the way, welcome to contribute left, right, inner join to IoTDB if you
> like ~ :D
>
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Xiangdong Huang <sa...@gmail.com> 于2021年3月8日周一 下午8:09写道：
>
> > Hi, yes, root is like "/" in Unix-like OS.
> >
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
> >
> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午6:14写道：
> >
> >> Hello Xiangdong,
> >>
> >> Thanks for confirming! One last question for now, all storage groups
> must
> >> start with "root" right? I cannot have storage groups like root1.demo,
> >> correct?
> >>
> >> Regards,
> >> dgargcs
> >>
> >> On Mon, 8 Mar 2021 at 15:37, Xiangdong Huang <sa...@gmail.com>
> wrote:
> >>
> >> > Hi Dhruv,
> >> >
> >> > Yes you are right. and IoTDB also supports join on timestamp across
> >> > storage groups.
> >> >
> >> > Q2: IoTDB does not support that now.
> >> > Current join looks like outer-join in RDBMS.
> >> > Supporting left join, right join and inner-join is not hard to
> >> implement,
> >> > just omit the rows that contain NULL.
> >> > (Better implementation will need CBO cost-based optimization, but
> IoTDB
> >> > lacks the statistics info for that.)
> >> >
> >> > Best,
> >> > -----------------------------------
> >> > Xiangdong Huang
> >> > School of Software, Tsinghua University
> >> >
> >> > 黄向东
> >> > 清华大学 软件学院
> >> >
> >> >
> >> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
> >> >
> >> > > Hello Xiangdong,
> >> > >
> >> > > Yes. Let me elaborate a bit about the join requirement. Suppose I
> >> want to
> >> > > do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2
> >> for
> >> > a
> >> > > common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It
> >> > should
> >> > > print out the last two rows from your table.
> >> > >
> >> > > In SQL, when we do a join, there are two different tables and then
> we
> >> do
> >> > a
> >> > > join based on some common column => root.demo.d1 is one table (has
> an
> >> > "m1"
> >> > > column) and another table root.demo.d2 (has an "m2" column). If I
> do a
> >> > join
> >> > > on d1 and d2 based on timestamp, it will return some rows containing
> >> m1
> >> > and
> >> > > m2 where timestamps are the same.
> >> > >
> >> > > I think the way to look at join queries in IoTDB is slightly
> >> different.
> >> > > Here, we have a storage group (root.demo) with different time-series
> >> > (d1.m1
> >> > > and d2.m2). For each insert in a storage group, I think IoTDB
> >> > automatically
> >> > > joins these time-series on timestamp since it assumes null values
> for
> >> > other
> >> > > time-series that were not given a value during insert. For example,
> >> if I
> >> > > do an "insert root.demo.d2(timestamp,m2) values
> (2021-08-03T00:00:00,
> >> > > 2021)", it will also append NULL value for root.demo.d1.m1 for this
> >> > > timestamp.
> >> > >
> >> > > Thus, to do a join within a storage group for *any number* of
> >> time-series
> >> > > is straightforward. I think this is trivially done by just giving
> the
> >> > time
> >> > > range and time-series names => "select d1.m1, d2.m2 from root.demo
> >> where
> >> > > timestamp >= 2019-19-11T00:00:00".
> >> > >
> >> > > Next, I thought about a similar query across storage groups, say
> >> > root.demo1
> >> > > and root.demo2. Here as well, I can run a similar query, but only
> >> start
> >> > > from a higher level in the storage hierarchy, i.e. "root" alone.
> >> "select
> >> > > demo1.m1, demo2.m2 from root where timestamp >=
> 2019-19-11T00:00:00".
> >> > >
> >> > > Q1. Is my interpretation of join on timestamp correct for IoTDB?
> >> > >
> >> > > Q2. Is there a way to omit rows that return one or more null column
> >> > values
> >> > > in the result?
> >> > >
> >> > > PS: Sorry for the long email. I am exploring IoTDB for a research
> >> project
> >> > > and just want to be clear in my understanding.
> >> > >
> >> > > Thanks in advance!
> >> > >
> >> > > Regards,
> >> > > dgargcs
> >> > >
> >> > > On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com>
> >> wrote:
> >> > >
> >> > > > Hi Dhruv,
> >> > > >
> >> > > > 1. Could you give me an example to describe what you want to
> query?
> >> > > > e.g., there are two time series:
> >> > > >
> >> > > > +-----------------------------+---------------+---------------+
> >> > > > | Time|root.demo.d1.m1|root.demo.d2.m2|
> >> > > > +-----------------------------+---------------+---------------+
> >> > > > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> >> > > > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> >> > > > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> >> > > > +-----------------------------+---------------+---------------+
> >> > > >
> >> > > > or,
> >> > > >
> >> > > > What you want to query? (you can consider it as a table in MySQL,
> >> and
> >> > > just
> >> > > > write the sqls you need in mysql.)
> >> > > >
> >> > > > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I
> >> > think.
> >> > > >
> >> > > > Best,
> >> > > > -----------------------------------
> >> > > > Xiangdong Huang
> >> > > > School of Software, Tsinghua University
> >> > > >
> >> > > > 黄向东
> >> > > > 清华大学 软件学院
> >> > > >
> >> > > >
> >> > > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> >> > > >
> >> > > > > Hello Xiangdong,
> >> > > > >
> >> > > > > Thanks for your response.
> >> > > > >
> >> > > > > 1. I have the same use-case: I want to run a join query on two
> >> time
> >> > > > > series but I think I am getting the syntax wrong.
> >> > > > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> >> > > > following
> >> > > > > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER,
> >> > > SLIMIT,
> >> > > > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> >> > > > > Could you please refer me to some documentation or provide a
> >> simple
> >> > > > example
> >> > > > > join query for IoTDB?
> >> > > > >
> >> > > > > 2. I am also excited to try out the cluster version. Is there an
> >> > > > > approximate timeline for v0.12 rollout?
> >> > > > >
> >> > > > > Thanks in advance.
> >> > > > >
> >> > > > > Regards,
> >> > > > > dgargcs
> >> > > > >
> >> > > > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <
> sainthxd@gmail.com>
> >> > > wrote:
> >> > > > >
> >> > > > > > Hi Dhruv,
> >> > > > > >
> >> > > > > > Welcome!
> >> > > > > >
> >> > > > > > 1. "join" for what requirements?
> >> > > > > >
> >> > > > > > IoTDB supports JOIN two time series ON their timestamp, i.e.,
> >> time
> >> > > > series
> >> > > > > > alignment.
> >> > > > > >
> >> > > > > > 2. I think the cluster version is coming... In the coming
> >> v0.12, I
> >> > > > think
> >> > > > > a
> >> > > > > > cluster version (that does
> >> > > > > > not support dynamically scale-out) will be released, and the
> >> > > scale-out
> >> > > > > > feature may be released in v0.13 or 0.14.
> >> > > > > >
> >> > > > > > Best,
> >> > > > > > -----------------------------------
> >> > > > > > Xiangdong Huang
> >> > > > > > School of Software, Tsinghua University
> >> > > > > >
> >> > > > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> >> > > > > >
> >> > > > > > > Hello,
> >> > > > > > >
> >> > > > > > > I am a researcher and a new user of IoTDB. I was excited
> when
> >> I
> >> > saw
> >> > > > the
> >> > > > > > > performance numbers of IoTDB compared to InfluxDB, and I
> have
> >> set
> >> > > it
> >> > > > up
> >> > > > > > on
> >> > > > > > > a few workstations and a Raspberry Pi device.
> >> > > > > > >
> >> > > > > > > I am becoming familiar with it and have a few queries:
> >> > > > > > >
> >> > > > > > > 1. Does IoTDB support Join queries? - I tried looking up
> >> some
> >> > > > > > > documentation on it, but could not find any. If the
> >> feature is
> >> > > > > > > available,
> >> > > > > > > could you please provide a sample query or the predicates
> >> for
> >> > > join
> >> > > > > > > queries?
> >> > > > > > > 2. I also browsed through the IoTDB Jira issues page. I
> saw
> >> > that
> >> > > > the
> >> > > > > > > distributed / cluster version is being developed.
> >> > Approximately
> >> > > > how
> >> > > > > > much
> >> > > > > > > time would it take to get a stable / working version of
> the
> >> > > > cluster
> >> > > > > > > IoTDB?
> >> > > > > > >
> >> > > > > > > It would be great if someone can respond to my queries. I
> >> would
> >> > be
> >> > > > > > > waiting for a reply. Thanks in advance.
> >> > > > > > >
> >> > > > > > > Regards,
> >> > > > > > > dgargcs
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: New user of IoTDB - a few queries
Posted by Xiangdong Huang <sa...@gmail.com>.
By the way, welcome to contribute left, right, inner join to IoTDB if you
like ~ :D
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Xiangdong Huang <sa...@gmail.com> 于2021年3月8日周一 下午8:09写道：
> Hi, yes, root is like "/" in Unix-like OS.
>
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午6:14写道：
>
>> Hello Xiangdong,
>>
>> Thanks for confirming! One last question for now, all storage groups must
>> start with "root" right? I cannot have storage groups like root1.demo,
>> correct?
>>
>> Regards,
>> dgargcs
>>
>> On Mon, 8 Mar 2021 at 15:37, Xiangdong Huang <sa...@gmail.com> wrote:
>>
>> > Hi Dhruv,
>> >
>> > Yes you are right. and IoTDB also supports join on timestamp across
>> > storage groups.
>> >
>> > Q2: IoTDB does not support that now.
>> > Current join looks like outer-join in RDBMS.
>> > Supporting left join, right join and inner-join is not hard to
>> implement,
>> > just omit the rows that contain NULL.
>> > (Better implementation will need CBO cost-based optimization, but IoTDB
>> > lacks the statistics info for that.)
>> >
>> > Best,
>> > -----------------------------------
>> > Xiangdong Huang
>> > School of Software, Tsinghua University
>> >
>> > 黄向东
>> > 清华大学 软件学院
>> >
>> >
>> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
>> >
>> > > Hello Xiangdong,
>> > >
>> > > Yes. Let me elaborate a bit about the join requirement. Suppose I
>> want to
>> > > do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2
>> for
>> > a
>> > > common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It
>> > should
>> > > print out the last two rows from your table.
>> > >
>> > > In SQL, when we do a join, there are two different tables and then we
>> do
>> > a
>> > > join based on some common column => root.demo.d1 is one table (has an
>> > "m1"
>> > > column) and another table root.demo.d2 (has an "m2" column). If I do a
>> > join
>> > > on d1 and d2 based on timestamp, it will return some rows containing
>> m1
>> > and
>> > > m2 where timestamps are the same.
>> > >
>> > > I think the way to look at join queries in IoTDB is slightly
>> different.
>> > > Here, we have a storage group (root.demo) with different time-series
>> > (d1.m1
>> > > and d2.m2). For each insert in a storage group, I think IoTDB
>> > automatically
>> > > joins these time-series on timestamp since it assumes null values for
>> > other
>> > > time-series that were not given a value during insert. For example,
>> if I
>> > > do an "insert root.demo.d2(timestamp,m2) values (2021-08-03T00:00:00,
>> > > 2021)", it will also append NULL value for root.demo.d1.m1 for this
>> > > timestamp.
>> > >
>> > > Thus, to do a join within a storage group for *any number* of
>> time-series
>> > > is straightforward. I think this is trivially done by just giving the
>> > time
>> > > range and time-series names => "select d1.m1, d2.m2 from root.demo
>> where
>> > > timestamp >= 2019-19-11T00:00:00".
>> > >
>> > > Next, I thought about a similar query across storage groups, say
>> > root.demo1
>> > > and root.demo2. Here as well, I can run a similar query, but only
>> start
>> > > from a higher level in the storage hierarchy, i.e. "root" alone.
>> "select
>> > > demo1.m1, demo2.m2 from root where timestamp >= 2019-19-11T00:00:00".
>> > >
>> > > Q1. Is my interpretation of join on timestamp correct for IoTDB?
>> > >
>> > > Q2. Is there a way to omit rows that return one or more null column
>> > values
>> > > in the result?
>> > >
>> > > PS: Sorry for the long email. I am exploring IoTDB for a research
>> project
>> > > and just want to be clear in my understanding.
>> > >
>> > > Thanks in advance!
>> > >
>> > > Regards,
>> > > dgargcs
>> > >
>> > > On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com>
>> wrote:
>> > >
>> > > > Hi Dhruv,
>> > > >
>> > > > 1. Could you give me an example to describe what you want to query?
>> > > > e.g., there are two time series:
>> > > >
>> > > > +-----------------------------+---------------+---------------+
>> > > > | Time|root.demo.d1.m1|root.demo.d2.m2|
>> > > > +-----------------------------+---------------+---------------+
>> > > > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
>> > > > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
>> > > > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
>> > > > +-----------------------------+---------------+---------------+
>> > > >
>> > > > or,
>> > > >
>> > > > What you want to query? (you can consider it as a table in MySQL,
>> and
>> > > just
>> > > > write the sqls you need in mysql.)
>> > > >
>> > > > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I
>> > think.
>> > > >
>> > > > Best,
>> > > > -----------------------------------
>> > > > Xiangdong Huang
>> > > > School of Software, Tsinghua University
>> > > >
>> > > > 黄向东
>> > > > 清华大学 软件学院
>> > > >
>> > > >
>> > > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
>> > > >
>> > > > > Hello Xiangdong,
>> > > > >
>> > > > > Thanks for your response.
>> > > > >
>> > > > > 1. I have the same use-case: I want to run a join query on two
>> time
>> > > > > series but I think I am getting the syntax wrong.
>> > > > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
>> > > > following
>> > > > > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER,
>> > > SLIMIT,
>> > > > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
>> > > > > Could you please refer me to some documentation or provide a
>> simple
>> > > > example
>> > > > > join query for IoTDB?
>> > > > >
>> > > > > 2. I am also excited to try out the cluster version. Is there an
>> > > > > approximate timeline for v0.12 rollout?
>> > > > >
>> > > > > Thanks in advance.
>> > > > >
>> > > > > Regards,
>> > > > > dgargcs
>> > > > >
>> > > > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > > > Hi Dhruv,
>> > > > > >
>> > > > > > Welcome!
>> > > > > >
>> > > > > > 1. "join" for what requirements?
>> > > > > >
>> > > > > > IoTDB supports JOIN two time series ON their timestamp, i.e.,
>> time
>> > > > series
>> > > > > > alignment.
>> > > > > >
>> > > > > > 2. I think the cluster version is coming... In the coming
>> v0.12, I
>> > > > think
>> > > > > a
>> > > > > > cluster version (that does
>> > > > > > not support dynamically scale-out) will be released, and the
>> > > scale-out
>> > > > > > feature may be released in v0.13 or 0.14.
>> > > > > >
>> > > > > > Best,
>> > > > > > -----------------------------------
>> > > > > > Xiangdong Huang
>> > > > > > School of Software, Tsinghua University
>> > > > > >
>> > > > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
>> > > > > >
>> > > > > > > Hello,
>> > > > > > >
>> > > > > > > I am a researcher and a new user of IoTDB. I was excited when
>> I
>> > saw
>> > > > the
>> > > > > > > performance numbers of IoTDB compared to InfluxDB, and I have
>> set
>> > > it
>> > > > up
>> > > > > > on
>> > > > > > > a few workstations and a Raspberry Pi device.
>> > > > > > >
>> > > > > > > I am becoming familiar with it and have a few queries:
>> > > > > > >
>> > > > > > > 1. Does IoTDB support Join queries? - I tried looking up
>> some
>> > > > > > > documentation on it, but could not find any. If the
>> feature is
>> > > > > > > available,
>> > > > > > > could you please provide a sample query or the predicates
>> for
>> > > join
>> > > > > > > queries?
>> > > > > > > 2. I also browsed through the IoTDB Jira issues page. I saw
>> > that
>> > > > the
>> > > > > > > distributed / cluster version is being developed.
>> > Approximately
>> > > > how
>> > > > > > much
>> > > > > > > time would it take to get a stable / working version of the
>> > > > cluster
>> > > > > > > IoTDB?
>> > > > > > >
>> > > > > > > It would be great if someone can respond to my queries. I
>> would
>> > be
>> > > > > > > waiting for a reply. Thanks in advance.
>> > > > > > >
>> > > > > > > Regards,
>> > > > > > > dgargcs
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: New user of IoTDB - a few queries
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi, yes, root is like "/" in Unix-like OS.
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午6:14写道：
> Hello Xiangdong,
>
> Thanks for confirming! One last question for now, all storage groups must
> start with "root" right? I cannot have storage groups like root1.demo,
> correct?
>
> Regards,
> dgargcs
>
> On Mon, 8 Mar 2021 at 15:37, Xiangdong Huang <sa...@gmail.com> wrote:
>
> > Hi Dhruv,
> >
> > Yes you are right. and IoTDB also supports join on timestamp across
> > storage groups.
> >
> > Q2: IoTDB does not support that now.
> > Current join looks like outer-join in RDBMS.
> > Supporting left join, right join and inner-join is not hard to implement,
> > just omit the rows that contain NULL.
> > (Better implementation will need CBO cost-based optimization, but IoTDB
> > lacks the statistics info for that.)
> >
> > Best,
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
> >
> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
> >
> > > Hello Xiangdong,
> > >
> > > Yes. Let me elaborate a bit about the join requirement. Suppose I want
> to
> > > do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2
> for
> > a
> > > common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It
> > should
> > > print out the last two rows from your table.
> > >
> > > In SQL, when we do a join, there are two different tables and then we
> do
> > a
> > > join based on some common column => root.demo.d1 is one table (has an
> > "m1"
> > > column) and another table root.demo.d2 (has an "m2" column). If I do a
> > join
> > > on d1 and d2 based on timestamp, it will return some rows containing m1
> > and
> > > m2 where timestamps are the same.
> > >
> > > I think the way to look at join queries in IoTDB is slightly different.
> > > Here, we have a storage group (root.demo) with different time-series
> > (d1.m1
> > > and d2.m2). For each insert in a storage group, I think IoTDB
> > automatically
> > > joins these time-series on timestamp since it assumes null values for
> > other
> > > time-series that were not given a value during insert. For example,
> if I
> > > do an "insert root.demo.d2(timestamp,m2) values (2021-08-03T00:00:00,
> > > 2021)", it will also append NULL value for root.demo.d1.m1 for this
> > > timestamp.
> > >
> > > Thus, to do a join within a storage group for *any number* of
> time-series
> > > is straightforward. I think this is trivially done by just giving the
> > time
> > > range and time-series names => "select d1.m1, d2.m2 from root.demo
> where
> > > timestamp >= 2019-19-11T00:00:00".
> > >
> > > Next, I thought about a similar query across storage groups, say
> > root.demo1
> > > and root.demo2. Here as well, I can run a similar query, but only start
> > > from a higher level in the storage hierarchy, i.e. "root" alone.
> "select
> > > demo1.m1, demo2.m2 from root where timestamp >= 2019-19-11T00:00:00".
> > >
> > > Q1. Is my interpretation of join on timestamp correct for IoTDB?
> > >
> > > Q2. Is there a way to omit rows that return one or more null column
> > values
> > > in the result?
> > >
> > > PS: Sorry for the long email. I am exploring IoTDB for a research
> project
> > > and just want to be clear in my understanding.
> > >
> > > Thanks in advance!
> > >
> > > Regards,
> > > dgargcs
> > >
> > > On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com>
> wrote:
> > >
> > > > Hi Dhruv,
> > > >
> > > > 1. Could you give me an example to describe what you want to query?
> > > > e.g., there are two time series:
> > > >
> > > > +-----------------------------+---------------+---------------+
> > > > | Time|root.demo.d1.m1|root.demo.d2.m2|
> > > > +-----------------------------+---------------+---------------+
> > > > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> > > > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> > > > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> > > > +-----------------------------+---------------+---------------+
> > > >
> > > > or,
> > > >
> > > > What you want to query? (you can consider it as a table in MySQL, and
> > > just
> > > > write the sqls you need in mysql.)
> > > >
> > > > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I
> > think.
> > > >
> > > > Best,
> > > > -----------------------------------
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > > 黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> > > >
> > > > > Hello Xiangdong,
> > > > >
> > > > > Thanks for your response.
> > > > >
> > > > > 1. I have the same use-case: I want to run a join query on two time
> > > > > series but I think I am getting the syntax wrong.
> > > > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> > > > following
> > > > > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER,
> > > SLIMIT,
> > > > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> > > > > Could you please refer me to some documentation or provide a simple
> > > > example
> > > > > join query for IoTDB?
> > > > >
> > > > > 2. I am also excited to try out the cluster version. Is there an
> > > > > approximate timeline for v0.12 rollout?
> > > > >
> > > > > Thanks in advance.
> > > > >
> > > > > Regards,
> > > > > dgargcs
> > > > >
> > > > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi Dhruv,
> > > > > >
> > > > > > Welcome!
> > > > > >
> > > > > > 1. "join" for what requirements?
> > > > > >
> > > > > > IoTDB supports JOIN two time series ON their timestamp, i.e.,
> time
> > > > series
> > > > > > alignment.
> > > > > >
> > > > > > 2. I think the cluster version is coming... In the coming v0.12,
> I
> > > > think
> > > > > a
> > > > > > cluster version (that does
> > > > > > not support dynamically scale-out) will be released, and the
> > > scale-out
> > > > > > feature may be released in v0.13 or 0.14.
> > > > > >
> > > > > > Best,
> > > > > > -----------------------------------
> > > > > > Xiangdong Huang
> > > > > > School of Software, Tsinghua University
> > > > > >
> > > > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I am a researcher and a new user of IoTDB. I was excited when I
> > saw
> > > > the
> > > > > > > performance numbers of IoTDB compared to InfluxDB, and I have
> set
> > > it
> > > > up
> > > > > > on
> > > > > > > a few workstations and a Raspberry Pi device.
> > > > > > >
> > > > > > > I am becoming familiar with it and have a few queries:
> > > > > > >
> > > > > > > 1. Does IoTDB support Join queries? - I tried looking up
> some
> > > > > > > documentation on it, but could not find any. If the feature
> is
> > > > > > > available,
> > > > > > > could you please provide a sample query or the predicates
> for
> > > join
> > > > > > > queries?
> > > > > > > 2. I also browsed through the IoTDB Jira issues page. I saw
> > that
> > > > the
> > > > > > > distributed / cluster version is being developed.
> > Approximately
> > > > how
> > > > > > much
> > > > > > > time would it take to get a stable / working version of the
> > > > cluster
> > > > > > > IoTDB?
> > > > > > >
> > > > > > > It would be great if someone can respond to my queries. I would
> > be
> > > > > > > waiting for a reply. Thanks in advance.
> > > > > > >
> > > > > > > Regards,
> > > > > > > dgargcs
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: New user of IoTDB - a few queries
Posted by Dhruv Garg <dg...@gmail.com>.
Hello Xiangdong,
Thanks for confirming! One last question for now, all storage groups must
start with "root" right? I cannot have storage groups like root1.demo,
correct?
Regards,
dgargcs
On Mon, 8 Mar 2021 at 15:37, Xiangdong Huang <sa...@gmail.com> wrote:
> Hi Dhruv,
>
> Yes you are right. and IoTDB also supports join on timestamp across
> storage groups.
>
> Q2: IoTDB does not support that now.
> Current join looks like outer-join in RDBMS.
> Supporting left join, right join and inner-join is not hard to implement,
> just omit the rows that contain NULL.
> (Better implementation will need CBO cost-based optimization, but IoTDB
> lacks the statistics info for that.)
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
>
> > Hello Xiangdong,
> >
> > Yes. Let me elaborate a bit about the join requirement. Suppose I want to
> > do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2 for
> a
> > common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It
> should
> > print out the last two rows from your table.
> >
> > In SQL, when we do a join, there are two different tables and then we do
> a
> > join based on some common column => root.demo.d1 is one table (has an
> "m1"
> > column) and another table root.demo.d2 (has an "m2" column). If I do a
> join
> > on d1 and d2 based on timestamp, it will return some rows containing m1
> and
> > m2 where timestamps are the same.
> >
> > I think the way to look at join queries in IoTDB is slightly different.
> > Here, we have a storage group (root.demo) with different time-series
> (d1.m1
> > and d2.m2). For each insert in a storage group, I think IoTDB
> automatically
> > joins these time-series on timestamp since it assumes null values for
> other
> > time-series that were not given a value during insert. For example, if I
> > do an "insert root.demo.d2(timestamp,m2) values (2021-08-03T00:00:00,
> > 2021)", it will also append NULL value for root.demo.d1.m1 for this
> > timestamp.
> >
> > Thus, to do a join within a storage group for *any number* of time-series
> > is straightforward. I think this is trivially done by just giving the
> time
> > range and time-series names => "select d1.m1, d2.m2 from root.demo where
> > timestamp >= 2019-19-11T00:00:00".
> >
> > Next, I thought about a similar query across storage groups, say
> root.demo1
> > and root.demo2. Here as well, I can run a similar query, but only start
> > from a higher level in the storage hierarchy, i.e. "root" alone. "select
> > demo1.m1, demo2.m2 from root where timestamp >= 2019-19-11T00:00:00".
> >
> > Q1. Is my interpretation of join on timestamp correct for IoTDB?
> >
> > Q2. Is there a way to omit rows that return one or more null column
> values
> > in the result?
> >
> > PS: Sorry for the long email. I am exploring IoTDB for a research project
> > and just want to be clear in my understanding.
> >
> > Thanks in advance!
> >
> > Regards,
> > dgargcs
> >
> > On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com> wrote:
> >
> > > Hi Dhruv,
> > >
> > > 1. Could you give me an example to describe what you want to query?
> > > e.g., there are two time series:
> > >
> > > +-----------------------------+---------------+---------------+
> > > | Time|root.demo.d1.m1|root.demo.d2.m2|
> > > +-----------------------------+---------------+---------------+
> > > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> > > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> > > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> > > +-----------------------------+---------------+---------------+
> > >
> > > or,
> > >
> > > What you want to query? (you can consider it as a table in MySQL, and
> > just
> > > write the sqls you need in mysql.)
> > >
> > > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I
> think.
> > >
> > > Best,
> > > -----------------------------------
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > > 黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> > >
> > > > Hello Xiangdong,
> > > >
> > > > Thanks for your response.
> > > >
> > > > 1. I have the same use-case: I want to run a join query on two time
> > > > series but I think I am getting the syntax wrong.
> > > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> > > following
> > > > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER,
> > SLIMIT,
> > > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> > > > Could you please refer me to some documentation or provide a simple
> > > example
> > > > join query for IoTDB?
> > > >
> > > > 2. I am also excited to try out the cluster version. Is there an
> > > > approximate timeline for v0.12 rollout?
> > > >
> > > > Thanks in advance.
> > > >
> > > > Regards,
> > > > dgargcs
> > > >
> > > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com>
> > wrote:
> > > >
> > > > > Hi Dhruv,
> > > > >
> > > > > Welcome!
> > > > >
> > > > > 1. "join" for what requirements?
> > > > >
> > > > > IoTDB supports JOIN two time series ON their timestamp, i.e., time
> > > series
> > > > > alignment.
> > > > >
> > > > > 2. I think the cluster version is coming... In the coming v0.12, I
> > > think
> > > > a
> > > > > cluster version (that does
> > > > > not support dynamically scale-out) will be released, and the
> > scale-out
> > > > > feature may be released in v0.13 or 0.14.
> > > > >
> > > > > Best,
> > > > > -----------------------------------
> > > > > Xiangdong Huang
> > > > > School of Software, Tsinghua University
> > > > >
> > > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I am a researcher and a new user of IoTDB. I was excited when I
> saw
> > > the
> > > > > > performance numbers of IoTDB compared to InfluxDB, and I have set
> > it
> > > up
> > > > > on
> > > > > > a few workstations and a Raspberry Pi device.
> > > > > >
> > > > > > I am becoming familiar with it and have a few queries:
> > > > > >
> > > > > > 1. Does IoTDB support Join queries? - I tried looking up some
> > > > > > documentation on it, but could not find any. If the feature is
> > > > > > available,
> > > > > > could you please provide a sample query or the predicates for
> > join
> > > > > > queries?
> > > > > > 2. I also browsed through the IoTDB Jira issues page. I saw
> that
> > > the
> > > > > > distributed / cluster version is being developed.
> Approximately
> > > how
> > > > > much
> > > > > > time would it take to get a stable / working version of the
> > > cluster
> > > > > > IoTDB?
> > > > > >
> > > > > > It would be great if someone can respond to my queries. I would
> be
> > > > > > waiting for a reply. Thanks in advance.
> > > > > >
> > > > > > Regards,
> > > > > > dgargcs
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: New user of IoTDB - a few queries
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Dhruv,
Yes you are right. and IoTDB also supports join on timestamp across
storage groups.
Q2: IoTDB does not support that now.
Current join looks like outer-join in RDBMS.
Supporting left join, right join and inner-join is not hard to implement,
just omit the rows that contain NULL.
(Better implementation will need CBO cost-based optimization, but IoTDB
lacks the statistics info for that.)
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午5:56写道：
> Hello Xiangdong,
>
> Yes. Let me elaborate a bit about the join requirement. Suppose I want to
> do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2 for a
> common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It should
> print out the last two rows from your table.
>
> In SQL, when we do a join, there are two different tables and then we do a
> join based on some common column => root.demo.d1 is one table (has an "m1"
> column) and another table root.demo.d2 (has an "m2" column). If I do a join
> on d1 and d2 based on timestamp, it will return some rows containing m1 and
> m2 where timestamps are the same.
>
> I think the way to look at join queries in IoTDB is slightly different.
> Here, we have a storage group (root.demo) with different time-series (d1.m1
> and d2.m2). For each insert in a storage group, I think IoTDB automatically
> joins these time-series on timestamp since it assumes null values for other
> time-series that were not given a value during insert. For example, if I
> do an "insert root.demo.d2(timestamp,m2) values (2021-08-03T00:00:00,
> 2021)", it will also append NULL value for root.demo.d1.m1 for this
> timestamp.
>
> Thus, to do a join within a storage group for *any number* of time-series
> is straightforward. I think this is trivially done by just giving the time
> range and time-series names => "select d1.m1, d2.m2 from root.demo where
> timestamp >= 2019-19-11T00:00:00".
>
> Next, I thought about a similar query across storage groups, say root.demo1
> and root.demo2. Here as well, I can run a similar query, but only start
> from a higher level in the storage hierarchy, i.e. "root" alone. "select
> demo1.m1, demo2.m2 from root where timestamp >= 2019-19-11T00:00:00".
>
> Q1. Is my interpretation of join on timestamp correct for IoTDB?
>
> Q2. Is there a way to omit rows that return one or more null column values
> in the result?
>
> PS: Sorry for the long email. I am exploring IoTDB for a research project
> and just want to be clear in my understanding.
>
> Thanks in advance!
>
> Regards,
> dgargcs
>
> On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com> wrote:
>
> > Hi Dhruv,
> >
> > 1. Could you give me an example to describe what you want to query?
> > e.g., there are two time series:
> >
> > +-----------------------------+---------------+---------------+
> > | Time|root.demo.d1.m1|root.demo.d2.m2|
> > +-----------------------------+---------------+---------------+
> > |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> > |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> > |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> > +-----------------------------+---------------+---------------+
> >
> > or,
> >
> > What you want to query? (you can consider it as a table in MySQL, and
> just
> > write the sqls you need in mysql.)
> >
> > 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I think.
> >
> > Best,
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
> >
> > Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> >
> > > Hello Xiangdong,
> > >
> > > Thanks for your response.
> > >
> > > 1. I have the same use-case: I want to run a join query on two time
> > > series but I think I am getting the syntax wrong.
> > > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> > following
> > > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER,
> SLIMIT,
> > > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> > > Could you please refer me to some documentation or provide a simple
> > example
> > > join query for IoTDB?
> > >
> > > 2. I am also excited to try out the cluster version. Is there an
> > > approximate timeline for v0.12 rollout?
> > >
> > > Thanks in advance.
> > >
> > > Regards,
> > > dgargcs
> > >
> > > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com>
> wrote:
> > >
> > > > Hi Dhruv,
> > > >
> > > > Welcome!
> > > >
> > > > 1. "join" for what requirements?
> > > >
> > > > IoTDB supports JOIN two time series ON their timestamp, i.e., time
> > series
> > > > alignment.
> > > >
> > > > 2. I think the cluster version is coming... In the coming v0.12, I
> > think
> > > a
> > > > cluster version (that does
> > > > not support dynamically scale-out) will be released, and the
> scale-out
> > > > feature may be released in v0.13 or 0.14.
> > > >
> > > > Best,
> > > > -----------------------------------
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> > > >
> > > > > Hello,
> > > > >
> > > > > I am a researcher and a new user of IoTDB. I was excited when I saw
> > the
> > > > > performance numbers of IoTDB compared to InfluxDB, and I have set
> it
> > up
> > > > on
> > > > > a few workstations and a Raspberry Pi device.
> > > > >
> > > > > I am becoming familiar with it and have a few queries:
> > > > >
> > > > > 1. Does IoTDB support Join queries? - I tried looking up some
> > > > > documentation on it, but could not find any. If the feature is
> > > > > available,
> > > > > could you please provide a sample query or the predicates for
> join
> > > > > queries?
> > > > > 2. I also browsed through the IoTDB Jira issues page. I saw that
> > the
> > > > > distributed / cluster version is being developed. Approximately
> > how
> > > > much
> > > > > time would it take to get a stable / working version of the
> > cluster
> > > > > IoTDB?
> > > > >
> > > > > It would be great if someone can respond to my queries. I would be
> > > > > waiting for a reply. Thanks in advance.
> > > > >
> > > > > Regards,
> > > > > dgargcs
> > > > >
> > > >
> > >
> >
>
Re: New user of IoTDB - a few queries
Posted by Dhruv Garg <dg...@gmail.com>.
Hello Xiangdong,
Yes. Let me elaborate a bit about the join requirement. Suppose I want to
do a join to print out values in root.demo.d1.m1 and root.demo.d2.m2 for a
common time range say 2019-12-11 00:00:00 to 2020-12-18 23:59:59. It should
print out the last two rows from your table.
In SQL, when we do a join, there are two different tables and then we do a
join based on some common column => root.demo.d1 is one table (has an "m1"
column) and another table root.demo.d2 (has an "m2" column). If I do a join
on d1 and d2 based on timestamp, it will return some rows containing m1 and
m2 where timestamps are the same.
I think the way to look at join queries in IoTDB is slightly different.
Here, we have a storage group (root.demo) with different time-series (d1.m1
and d2.m2). For each insert in a storage group, I think IoTDB automatically
joins these time-series on timestamp since it assumes null values for other
time-series that were not given a value during insert. For example, if I
do an "insert root.demo.d2(timestamp,m2) values (2021-08-03T00:00:00,
2021)", it will also append NULL value for root.demo.d1.m1 for this
timestamp.
Thus, to do a join within a storage group for *any number* of time-series
is straightforward. I think this is trivially done by just giving the time
range and time-series names => "select d1.m1, d2.m2 from root.demo where
timestamp >= 2019-19-11T00:00:00".
Next, I thought about a similar query across storage groups, say root.demo1
and root.demo2. Here as well, I can run a similar query, but only start
from a higher level in the storage hierarchy, i.e. "root" alone. "select
demo1.m1, demo2.m2 from root where timestamp >= 2019-19-11T00:00:00".
Q1. Is my interpretation of join on timestamp correct for IoTDB?
Q2. Is there a way to omit rows that return one or more null column values
in the result?
PS: Sorry for the long email. I am exploring IoTDB for a research project
and just want to be clear in my understanding.
Thanks in advance!
Regards,
dgargcs
On Mon, 8 Mar 2021 at 14:12, Xiangdong Huang <sa...@gmail.com> wrote:
> Hi Dhruv,
>
> 1. Could you give me an example to describe what you want to query?
> e.g., there are two time series:
>
> +-----------------------------+---------------+---------------+
> | Time|root.demo.d1.m1|root.demo.d2.m2|
> +-----------------------------+---------------+---------------+
> |1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
> |2019-12-11T10:00:00.000+08:00| 1.0| NULL|
> |2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
> +-----------------------------+---------------+---------------+
>
> or,
>
> What you want to query? (you can consider it as a table in MySQL, and just
> write the sqls you need in mysql.)
>
> 2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I think.
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
>
> > Hello Xiangdong,
> >
> > Thanks for your response.
> >
> > 1. I have the same use-case: I want to run a join query on two time
> > series but I think I am getting the syntax wrong.
> > I tried a few SQL-like join queries on IoTDB v0.11.2 and got the
> following
> > error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER, SLIMIT,
> > LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> > Could you please refer me to some documentation or provide a simple
> example
> > join query for IoTDB?
> >
> > 2. I am also excited to try out the cluster version. Is there an
> > approximate timeline for v0.12 rollout?
> >
> > Thanks in advance.
> >
> > Regards,
> > dgargcs
> >
> > On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com> wrote:
> >
> > > Hi Dhruv,
> > >
> > > Welcome!
> > >
> > > 1. "join" for what requirements?
> > >
> > > IoTDB supports JOIN two time series ON their timestamp, i.e., time
> series
> > > alignment.
> > >
> > > 2. I think the cluster version is coming... In the coming v0.12, I
> think
> > a
> > > cluster version (that does
> > > not support dynamically scale-out) will be released, and the scale-out
> > > feature may be released in v0.13 or 0.14.
> > >
> > > Best,
> > > -----------------------------------
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> > >
> > > > Hello,
> > > >
> > > > I am a researcher and a new user of IoTDB. I was excited when I saw
> the
> > > > performance numbers of IoTDB compared to InfluxDB, and I have set it
> up
> > > on
> > > > a few workstations and a Raspberry Pi device.
> > > >
> > > > I am becoming familiar with it and have a few queries:
> > > >
> > > > 1. Does IoTDB support Join queries? - I tried looking up some
> > > > documentation on it, but could not find any. If the feature is
> > > > available,
> > > > could you please provide a sample query or the predicates for join
> > > > queries?
> > > > 2. I also browsed through the IoTDB Jira issues page. I saw that
> the
> > > > distributed / cluster version is being developed. Approximately
> how
> > > much
> > > > time would it take to get a stable / working version of the
> cluster
> > > > IoTDB?
> > > >
> > > > It would be great if someone can respond to my queries. I would be
> > > > waiting for a reply. Thanks in advance.
> > > >
> > > > Regards,
> > > > dgargcs
> > > >
> > >
> >
>
Re: New user of IoTDB - a few queries
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Dhruv,
1. Could you give me an example to describe what you want to query?
e.g., there are two time series:
+-----------------------------+---------------+---------------+
| Time|root.demo.d1.m1|root.demo.d2.m2|
+-----------------------------+---------------+---------------+
|1970-01-01T08:00:00.001+08:00| 1.0| 2.0|
|2019-12-11T10:00:00.000+08:00| 1.0| NULL|
|2020-12-18T11:45:14.519+08:00| 2.0| 2.0|
+-----------------------------+---------------+---------------+
or,
What you want to query? (you can consider it as a table in MySQL, and just
write the sqls you need in mysql.)
2. Ah, we are testing the performance of v0.12. maybe 2-3 weeks, I think.
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Dhruv Garg <dg...@gmail.com> 于2021年3月8日周一 下午4:11写道：
> Hello Xiangdong,
>
> Thanks for your response.
>
> 1. I have the same use-case: I want to run a join query on two time
> series but I think I am getting the syntax wrong.
> I tried a few SQL-like join queries on IoTDB v0.11.2 and got the following
> error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER, SLIMIT,
> LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
> Could you please refer me to some documentation or provide a simple example
> join query for IoTDB?
>
> 2. I am also excited to try out the cluster version. Is there an
> approximate timeline for v0.12 rollout?
>
> Thanks in advance.
>
> Regards,
> dgargcs
>
> On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com> wrote:
>
> > Hi Dhruv,
> >
> > Welcome!
> >
> > 1. "join" for what requirements?
> >
> > IoTDB supports JOIN two time series ON their timestamp, i.e., time series
> > alignment.
> >
> > 2. I think the cluster version is coming... In the coming v0.12, I think
> a
> > cluster version (that does
> > not support dynamically scale-out) will be released, and the scale-out
> > feature may be released in v0.13 or 0.14.
> >
> > Best,
> > -----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> >
> > > Hello,
> > >
> > > I am a researcher and a new user of IoTDB. I was excited when I saw the
> > > performance numbers of IoTDB compared to InfluxDB, and I have set it up
> > on
> > > a few workstations and a Raspberry Pi device.
> > >
> > > I am becoming familiar with it and have a few queries:
> > >
> > > 1. Does IoTDB support Join queries? - I tried looking up some
> > > documentation on it, but could not find any. If the feature is
> > > available,
> > > could you please provide a sample query or the predicates for join
> > > queries?
> > > 2. I also browsed through the IoTDB Jira issues page. I saw that the
> > > distributed / cluster version is being developed. Approximately how
> > much
> > > time would it take to get a stable / working version of the cluster
> > > IoTDB?
> > >
> > > It would be great if someone can respond to my queries. I would be
> > > waiting for a reply. Thanks in advance.
> > >
> > > Regards,
> > > dgargcs
> > >
> >
>
Re: New user of IoTDB - a few queries
Posted by Dhruv Garg <dg...@gmail.com>.
Hello Xiangdong,
Thanks for your response.
1. I have the same use-case: I want to run a join query on two time
series but I think I am getting the syntax wrong.
I tried a few SQL-like join queries on IoTDB v0.11.2 and got the following
error: "mismatched input 'JOIN' expecting {<EOF>, WHERE, ORDER, SLIMIT,
LIMIT, OFFSET, SOFFSET, FILL, GROUP, DISABLE, ALIGN, ',', '.'}"
Could you please refer me to some documentation or provide a simple example
join query for IoTDB?
2. I am also excited to try out the cluster version. Is there an
approximate timeline for v0.12 rollout?
Thanks in advance.
Regards,
dgargcs
On Tue, 2 Mar 2021 at 17:24, Xiangdong Huang <sa...@gmail.com> wrote:
> Hi Dhruv,
>
> Welcome!
>
> 1. "join" for what requirements?
>
> IoTDB supports JOIN two time series ON their timestamp, i.e., time series
> alignment.
>
> 2. I think the cluster version is coming... In the coming v0.12, I think a
> cluster version (that does
> not support dynamically scale-out) will be released, and the scale-out
> feature may be released in v0.13 or 0.14.
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
>
> > Hello,
> >
> > I am a researcher and a new user of IoTDB. I was excited when I saw the
> > performance numbers of IoTDB compared to InfluxDB, and I have set it up
> on
> > a few workstations and a Raspberry Pi device.
> >
> > I am becoming familiar with it and have a few queries:
> >
> > 1. Does IoTDB support Join queries? - I tried looking up some
> > documentation on it, but could not find any. If the feature is
> > available,
> > could you please provide a sample query or the predicates for join
> > queries?
> > 2. I also browsed through the IoTDB Jira issues page. I saw that the
> > distributed / cluster version is being developed. Approximately how
> much
> > time would it take to get a stable / working version of the cluster
> > IoTDB?
> >
> > It would be great if someone can respond to my queries. I would be
> > waiting for a reply. Thanks in advance.
> >
> > Regards,
> > dgargcs
> >
>
Re: New user of IoTDB - a few queries
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi Dhruv,
Welcome!
1. "join" for what requirements?
IoTDB supports JOIN two time series ON their timestamp, i.e., time series
alignment.
2. I think the cluster version is coming... In the coming v0.12, I think a
cluster version (that does
not support dynamically scale-out) will be released, and the scale-out
feature may be released in v0.13 or 0.14.
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
Dhruv Garg <dg...@gmail.com> 于2021年3月2日周二 下午3:54写道：
> Hello,
>
> I am a researcher and a new user of IoTDB. I was excited when I saw the
> performance numbers of IoTDB compared to InfluxDB, and I have set it up on
> a few workstations and a Raspberry Pi device.
>
> I am becoming familiar with it and have a few queries:
>
> 1. Does IoTDB support Join queries? - I tried looking up some
> documentation on it, but could not find any. If the feature is
> available,
> could you please provide a sample query or the predicates for join
> queries?
> 2. I also browsed through the IoTDB Jira issues page. I saw that the
> distributed / cluster version is being developed. Approximately how much
> time would it take to get a stable / working version of the cluster
> IoTDB?
>
> It would be great if someone can respond to my queries. I would be
> waiting for a reply. Thanks in advance.
>
> Regards,
> dgargcs
>