You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by "Xingjun Hao (Jira)" <ji...@apache.org> on 2020/02/13 07:41:00 UTC

[jira] [Created] (CARBONDATA-3700) Optimize prune performance when prunning with multi-threads

Xingjun Hao created CARBONDATA-3700:
---------------------------------------

             Summary: Optimize prune performance when prunning with multi-threads
                 Key: CARBONDATA-3700
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3700
             Project: CarbonData
          Issue Type: Bug
          Components: data-query
    Affects Versions: 2.0.0
            Reporter: Xingjun Hao


When pruning with multi-threads, there is a bug hambers the prunning performance heavily. 

When the datamap pruning results in no blocklet to map filter, The getExtendblocklet function aims to get the extend blocklet metadata, when the Input is a empty blocklet list, this function should return a extend blocklet list directyly , but now there is a bug leading to a hashset add operation overhead.

Meanwhile ,When pruning with multi-threads, the getExtendblocklet function will be triggerd for each blocklet. This should avoided by trgger this function for each segment.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)