[jira] [Created] (CARBONDATA-3481) Multi-thread pruning fails when datamaps count is just near numOfThreadsForPruning

["Ajantha Bhat (JIRA)" <ji...@apache.org>]

Ajantha Bhat created CARBONDATA-3481:
----------------------------------------

             Summary: Multi-thread pruning fails when datamaps count is just near numOfThreadsForPruning
                 Key: CARBONDATA-3481
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3481
             Project: CarbonData
          Issue Type: Bug
            Reporter: Ajantha Bhat


Problem : Multi-thread pruning fails when datamaps count is just near numOfThreadsForPruning.

Cause : When the datamaps count is just near numOfThreadsForPruning,
As code is checking '>= ', last thread may not get the datamaps for prune. Hence array out of index exception is thrown in this scenario.
There is no issues with higher number of datamaps.

solution: In this scenario launch threads based on the distribution value, not on the hardcoded value

 

steps to reproduce:

5 index files with total 0.32 million data file (each has 50k, 120k, 50k, 60k, 50k each)

with default thread count as 4, array of bound index is observed for below line.

final List<SegmentDataMapGroup> segmentDataMapGroups = datamapListForEachThread.get(i);



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)