[jira] [Commented] (IOTDB-471) NullPoint exception when merging all TsFiles (may caused by TimeRange Partition)

["Jialin Qiao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IOTDB-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036170#comment-17036170 ] 

Jialin Qiao commented on IOTDB-471:
-----------------------------------

We do not figure out how it occurs, only add some log. So this issue is not closed now.

> NullPoint exception when merging all TsFiles (may caused by TimeRange Partition)
> --------------------------------------------------------------------------------
>
>                 Key: IOTDB-471
>                 URL: https://issues.apache.org/jira/browse/IOTDB-471
>             Project: Apache IoTDB
>          Issue Type: Bug
>          Components: Core/Engine
>            Reporter: Xiangdong Huang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.10.0-SNAPSHOT
>
>         Attachments: image-2020-02-10-20-32-03-335.png
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> Hi,
> In an application, there are millions of time series. In this case, when we enable dynamic_parameter, each TsFile has only 7MB.
> Therefore, there are too many TsFiles and the following query is quite slow:
> SELECT longitude,latitude,altitude,'11',windDirection2min,...(other 65 columns) FROM root.national.*.5.* WHERE time = 2020-01-18 20:00:00 group by device
> The above query returns 2400 rows and spends 8 seconds....... (Total data is just 2GB).
> So, I begin to run merge to accelerate the query speed and the following exception occurs:
> !image-2020-02-10-20-32-03-335.png|width=799,height=527!
> If we check the codes, the null point exception is caused by:
>  
> {code:java}
>  
> private boolean updateLatestFlushTimeCallback(TsFileProcessor processor) {
>   // update the largest timestamp in the last flushing memtable
>   for (Entry<String, Long> entry : latestTimeForEachDevice.get(processor.getTimeRangeId())
>       .entrySet()) {
>     latestFlushedTimeForEachDevice.get(processor.getTimeRangeId())
>         .put(entry.getKey(), entry.getValue());
>   }
>   return true;
> }
>  
> {code}
>  
> I know it is hard to reproduce, but can someone help to check and analyze the codes?
> Or, at least add more logs for helping to analyze such cases in the future.
> [~SilverNarcissus] [~jt2594838]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)