[jira] [Commented] (IOTDB-368) Special design for time series which has high frequency (need to discuss the user scenario)

["Dongming Zhao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IOTDB-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527282#comment-17527282 ] 

Dongming Zhao commented on IOTDB-368:
-------------------------------------

Is the idea of (2) similar to Rollup(in OpenTSDB and KairosDB) or DownSampling(in IoTDB)? All these above is providing optional queries with extra storage cost but better performance(for special queries). Native aggregation with down sampling is already supported in IoTDB, the only problem is online computing may be much slower than pre-aggregation in Rollup.

Besides, Gorilla for Facebook is doing it much further, where historical data is dropped and replaced with down sampling results. But apparently, this is for special scenarios where exact historical data is hardly accessed and information loss is acceptable with storage space saving.

Hopefully these may be helpful.

> Special design for time series which has  high frequency  (need to discuss the user scenario)
> ---------------------------------------------------------------------------------------------
>
>                 Key: IOTDB-368
>                 URL: https://issues.apache.org/jira/browse/IOTDB-368
>             Project: Apache IoTDB
>          Issue Type: New Feature
>            Reporter: Xiangdong Huang
>            Priority: Major
>
> Hi, recently I find some users want to store time series with high frequency, e.g., 100KHz.
>  
> As far as I know, IoTDB can support the above kind of frequency by using batch insertion (with 1 client).
>  
> However, I think we can do it better:
> (1) considering better compression (both for the timestamp and the values)
> (2) In this case, does user really need to read data one point by one point?  Running a special query on this kind of data may be helpful.
>  
> More discussion is needed.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)