You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Jialin Qiao <qj...@mails.tsinghua.edu.cn> on 2019/09/07 02:29:36 UTC

A new result set format

Hi,

As described in this issue, a new result set format is wanted by users. I'd like to open a discussion here.

For simplicity, I refer this format "time, root.sg1.d1.s1, root.sg1.d2.s1" to wide table, and "time, deviceId, s1" as narrow table. 

This issue is not only about how to organize the results, but also the query process. 

There are some advantages about narrow table.

(1) For wide table, we need to open a SeriesReader for each series at the same time, each SeriesReader holds some ChunkMetadatas. For narrow table, we only need to open SeriesReaders for one device at one time, then return results and open SeriesReaders for the next device, which occupies less memory compared to the wide table. 
(2) Avoid reading all series at once may also improve the query latency.

There is also a question:

(1) If we show result in the narrow table format for users, do we need to highlight the concept of table and device? 
(2) If the answer of the first question is yes, do we need to support sql: "select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? This may involve a lot of work...

From my side, I prefer the answers of the two questions are all NO. Then we do not need to change the sql grammar and only use a new query process to organize the result set.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -----原始邮件-----
> 发件人: "Jialin Qiao (Jira)" <ji...@apache.org>
> 发送时间: 2019-09-07 09:40:00 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-203) A new result set format
> 
> Jialin Qiao created IOTDB-203:
> ---------------------------------
> 
>              Summary: A new result set format
>                  Key: IOTDB-203
>                  URL: https://issues.apache.org/jira/browse/IOTDB-203
>              Project: Apache IoTDB
>           Issue Type: New Feature
>             Reporter: Jialin Qiao
> 
> 
> When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default result set format in IoTDB is 
> 
> "time, root.sg1.d1.s1, root.sg1.d2.s1"
> 
> 1 , 1, 1
> 
> 2, 2, 2
> 
> However, some users want to get another format, The results could be grouped by device, then sorted by time.
> 
> "time, deviceId, s1".
> 
> 1, root.sg1.d1, 1
> 
> 2, root.sg1.d2, 2
> 
>  
> 
> This can be done in the client, but it would be better if we support this format in the server.
> 
>  
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)

Re: A new result set format

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi Jialin,

well thats no hard requirement... ist totally fine if things come in that way from time to time : )
As I'm so new to IoTDB I'm just still trying to understand whats the user group currently and who drives feature ideas and stuff : )

I hope that we start soon with using IoTDB and then we'll very likely also contribute some wishes : )

Julian

Am 07.09.19, 09:26 schrieb "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>:

    Hi Julian,
    
    He is my friend and contacted me offline, because I advertise IoTDB in my weChat(like facebook or twitter). 
    
    Next time I will try to let him put issue in the mail list himself :)
    
    Best,
    --
    Jialin Qiao
    School of Software, Tsinghua University
    
    乔嘉林
    清华大学 软件学院
    
    > -----原始邮件-----
    > 发件人: "Julian Feinauer" <j....@pragmaticminds.de>
    > 发送时间: 2019-09-07 13:52:17 (星期六)
    > 收件人: "dev@iotdb.apache.org" <de...@iotdb.apache.org>
    > 抄送: 
    > 主题: Re: A new result set format
    > 
    > Hi Jialin,
    > 
    > perhaps one question about "wanted by users" means (as I didn’t see anything on the list).
    > How do these users get in contact with you?
    > 
    > Julian
    > 
    > Am 07.09.19, 04:29 schrieb "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>:
    > 
    >     Hi,
    >     
    >     As described in this issue, a new result set format is wanted by users. I'd like to open a discussion here.
    >     
    >     For simplicity, I refer this format "time, root.sg1.d1.s1, root.sg1.d2.s1" to wide table, and "time, deviceId, s1" as narrow table. 
    >     
    >     This issue is not only about how to organize the results, but also the query process. 
    >     
    >     There are some advantages about narrow table.
    >     
    >     (1) For wide table, we need to open a SeriesReader for each series at the same time, each SeriesReader holds some ChunkMetadatas. For narrow table, we only need to open SeriesReaders for one device at one time, then return results and open SeriesReaders for the next device, which occupies less memory compared to the wide table. 
    >     (2) Avoid reading all series at once may also improve the query latency.
    >     
    >     There is also a question:
    >     
    >     (1) If we show result in the narrow table format for users, do we need to highlight the concept of table and device? 
    >     (2) If the answer of the first question is yes, do we need to support sql: "select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? This may involve a lot of work...
    >     
    >     From my side, I prefer the answers of the two questions are all NO. Then we do not need to change the sql grammar and only use a new query process to organize the result set.
    >     
    >     Best,
    >     --
    >     Jialin Qiao
    >     School of Software, Tsinghua University
    >     
    >     乔嘉林
    >     清华大学 软件学院
    >     
    >     > -----原始邮件-----
    >     > 发件人: "Jialin Qiao (Jira)" <ji...@apache.org>
    >     > 发送时间: 2019-09-07 09:40:00 (星期六)
    >     > 收件人: dev@iotdb.apache.org
    >     > 抄送: 
    >     > 主题: [jira] [Created] (IOTDB-203) A new result set format
    >     > 
    >     > Jialin Qiao created IOTDB-203:
    >     > ---------------------------------
    >     > 
    >     >              Summary: A new result set format
    >     >                  Key: IOTDB-203
    >     >                  URL: https://issues.apache.org/jira/browse/IOTDB-203
    >     >              Project: Apache IoTDB
    >     >           Issue Type: New Feature
    >     >             Reporter: Jialin Qiao
    >     > 
    >     > 
    >     > When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default result set format in IoTDB is 
    >     > 
    >     > "time, root.sg1.d1.s1, root.sg1.d2.s1"
    >     > 
    >     > 1 , 1, 1
    >     > 
    >     > 2, 2, 2
    >     > 
    >     > However, some users want to get another format, The results could be grouped by device, then sorted by time.
    >     > 
    >     > "time, deviceId, s1".
    >     > 
    >     > 1, root.sg1.d1, 1
    >     > 
    >     > 2, root.sg1.d2, 2
    >     > 
    >     >  
    >     > 
    >     > This can be done in the client, but it would be better if we support this format in the server.
    >     > 
    >     >  
    >     > 
    >     > 
    >     > 
    >     > --
    >     > This message was sent by Atlassian Jira
    >     > (v8.3.2#803003)
    >     
    > 
    


Re: A new result set format

Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Hi Julian,

He is my friend and contacted me offline, because I advertise IoTDB in my weChat(like facebook or twitter). 

Next time I will try to let him put issue in the mail list himself :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -----原始邮件-----
> 发件人: "Julian Feinauer" <j....@pragmaticminds.de>
> 发送时间: 2019-09-07 13:52:17 (星期六)
> 收件人: "dev@iotdb.apache.org" <de...@iotdb.apache.org>
> 抄送: 
> 主题: Re: A new result set format
> 
> Hi Jialin,
> 
> perhaps one question about "wanted by users" means (as I didn’t see anything on the list).
> How do these users get in contact with you?
> 
> Julian
> 
> Am 07.09.19, 04:29 schrieb "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>:
> 
>     Hi,
>     
>     As described in this issue, a new result set format is wanted by users. I'd like to open a discussion here.
>     
>     For simplicity, I refer this format "time, root.sg1.d1.s1, root.sg1.d2.s1" to wide table, and "time, deviceId, s1" as narrow table. 
>     
>     This issue is not only about how to organize the results, but also the query process. 
>     
>     There are some advantages about narrow table.
>     
>     (1) For wide table, we need to open a SeriesReader for each series at the same time, each SeriesReader holds some ChunkMetadatas. For narrow table, we only need to open SeriesReaders for one device at one time, then return results and open SeriesReaders for the next device, which occupies less memory compared to the wide table. 
>     (2) Avoid reading all series at once may also improve the query latency.
>     
>     There is also a question:
>     
>     (1) If we show result in the narrow table format for users, do we need to highlight the concept of table and device? 
>     (2) If the answer of the first question is yes, do we need to support sql: "select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? This may involve a lot of work...
>     
>     From my side, I prefer the answers of the two questions are all NO. Then we do not need to change the sql grammar and only use a new query process to organize the result set.
>     
>     Best,
>     --
>     Jialin Qiao
>     School of Software, Tsinghua University
>     
>     乔嘉林
>     清华大学 软件学院
>     
>     > -----原始邮件-----
>     > 发件人: "Jialin Qiao (Jira)" <ji...@apache.org>
>     > 发送时间: 2019-09-07 09:40:00 (星期六)
>     > 收件人: dev@iotdb.apache.org
>     > 抄送: 
>     > 主题: [jira] [Created] (IOTDB-203) A new result set format
>     > 
>     > Jialin Qiao created IOTDB-203:
>     > ---------------------------------
>     > 
>     >              Summary: A new result set format
>     >                  Key: IOTDB-203
>     >                  URL: https://issues.apache.org/jira/browse/IOTDB-203
>     >              Project: Apache IoTDB
>     >           Issue Type: New Feature
>     >             Reporter: Jialin Qiao
>     > 
>     > 
>     > When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default result set format in IoTDB is 
>     > 
>     > "time, root.sg1.d1.s1, root.sg1.d2.s1"
>     > 
>     > 1 , 1, 1
>     > 
>     > 2, 2, 2
>     > 
>     > However, some users want to get another format, The results could be grouped by device, then sorted by time.
>     > 
>     > "time, deviceId, s1".
>     > 
>     > 1, root.sg1.d1, 1
>     > 
>     > 2, root.sg1.d2, 2
>     > 
>     >  
>     > 
>     > This can be done in the client, but it would be better if we support this format in the server.
>     > 
>     >  
>     > 
>     > 
>     > 
>     > --
>     > This message was sent by Atlassian Jira
>     > (v8.3.2#803003)
>     
> 

Re: A new result set format

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi Jialin,

perhaps one question about "wanted by users" means (as I didn’t see anything on the list).
How do these users get in contact with you?

Julian

Am 07.09.19, 04:29 schrieb "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>:

    Hi,
    
    As described in this issue, a new result set format is wanted by users. I'd like to open a discussion here.
    
    For simplicity, I refer this format "time, root.sg1.d1.s1, root.sg1.d2.s1" to wide table, and "time, deviceId, s1" as narrow table. 
    
    This issue is not only about how to organize the results, but also the query process. 
    
    There are some advantages about narrow table.
    
    (1) For wide table, we need to open a SeriesReader for each series at the same time, each SeriesReader holds some ChunkMetadatas. For narrow table, we only need to open SeriesReaders for one device at one time, then return results and open SeriesReaders for the next device, which occupies less memory compared to the wide table. 
    (2) Avoid reading all series at once may also improve the query latency.
    
    There is also a question:
    
    (1) If we show result in the narrow table format for users, do we need to highlight the concept of table and device? 
    (2) If the answer of the first question is yes, do we need to support sql: "select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? This may involve a lot of work...
    
    From my side, I prefer the answers of the two questions are all NO. Then we do not need to change the sql grammar and only use a new query process to organize the result set.
    
    Best,
    --
    Jialin Qiao
    School of Software, Tsinghua University
    
    乔嘉林
    清华大学 软件学院
    
    > -----原始邮件-----
    > 发件人: "Jialin Qiao (Jira)" <ji...@apache.org>
    > 发送时间: 2019-09-07 09:40:00 (星期六)
    > 收件人: dev@iotdb.apache.org
    > 抄送: 
    > 主题: [jira] [Created] (IOTDB-203) A new result set format
    > 
    > Jialin Qiao created IOTDB-203:
    > ---------------------------------
    > 
    >              Summary: A new result set format
    >                  Key: IOTDB-203
    >                  URL: https://issues.apache.org/jira/browse/IOTDB-203
    >              Project: Apache IoTDB
    >           Issue Type: New Feature
    >             Reporter: Jialin Qiao
    > 
    > 
    > When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default result set format in IoTDB is 
    > 
    > "time, root.sg1.d1.s1, root.sg1.d2.s1"
    > 
    > 1 , 1, 1
    > 
    > 2, 2, 2
    > 
    > However, some users want to get another format, The results could be grouped by device, then sorted by time.
    > 
    > "time, deviceId, s1".
    > 
    > 1, root.sg1.d1, 1
    > 
    > 2, root.sg1.d2, 2
    > 
    >  
    > 
    > This can be done in the client, but it would be better if we support this format in the server.
    > 
    >  
    > 
    > 
    > 
    > --
    > This message was sent by Atlassian Jira
    > (v8.3.2#803003)