[jira] [Commented] (DRILL-5395) Query on MapR-DB table fails with NPE due to an issue with assignment logic

["Padma Penumarthy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946511#comment-15946511 ] 

Padma Penumarthy commented on DRILL-5395:
-----------------------------------------

Based on number of fragments (decided by parallelization) and HBase regions to assign to these fragments, we calculate minimum and maximum regions to assign per fragment. 
Each region is assigned to a fragment on the node it is located on, fragments being picked in a round robin fashion. At the end, we make adjustments so each fragment is assigned at least the minimum and not more than maximum regions per fragment.
We can  have a case where we assigned up to minimum per fragment and still have regions to assign. In that case, minHeap will be null. We try to assign the unassigned regions to fragments with less than minimum by accessing minHeap, which is null and thus causing NPE.

In this case, 
we have 5 regions with ~300K rows. We apply selectivity of 0.5 and determine the cost as ~150,000. With slice target of 100K, we  create 2 fragments. On a 3 node cluster, we have 2 fragments assigned to 2 nodes and none assigned to 3rd node. With 5 regions to assign, minimum regions to assign per fragment is calculated as 5/2 i.e. 2 and maximum regions to assign per fragment is 3.  In the first round,  each region is allocated to a fragment on the node on which it is located. 4 regions got assigned to 2 fragments i.e. both the fragments have 2 each. One of the regions is located on the node which does not have any fragments and it does not get assigned to a fragment in the first round. Once initial assignment based on region location is done, we try to adjust the assignments so that each fragment gets assigned at least the minimum and at most the maximum. For this adjustment phase, we build a minHeap which is supposed to contain fragments which have less than minimum assigned. In this case, since we assigned 2 regions each to 2 fragments, minHeap is null. For assigning the 5th region, we try to get fragment which has least number of regions assigned by accessing minHeap, which is NULL.  This causes Null Pointer Exception.

Fix is to include the fragments which have minimum assigned in minHeap (not just less than minimum).




> Query on MapR-DB table fails with NPE due to an issue with assignment logic
> ---------------------------------------------------------------------------
>
>                 Key: DRILL-5395
>                 URL: https://issues.apache.org/jira/browse/DRILL-5395
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization, Storage - MapRDB
>    Affects Versions: 1.9.0, 1.10.0
>            Reporter: Abhishek Girish
>            Assignee: Padma Penumarthy
>              Labels: MapR-DB-Binary
>             Fix For: 1.11.0
>
>
> We uncovered this issue when working on DRILL-5394. 
> The MapR-DB table in question had 5 tablets with skewed data distribution (~6 million rows). A partial WIP fix for DRILL-5394 caused the number of rows to be reported incorrectly (~300,000). 2 minor fragments were created (due to filter selectivity) for scanning the 5 tablets. And this resulted in an NPE, possibly related to an issue with assignment logic, that was now exposed. 
> Representative query:
> {code}
> SELECT Convert_from(avail.customer, 'UTF8') AS ABC, 
>        Convert_from(prop.customer, 'UTF8')  AS PQR 
> FROM   (SELECT Convert_from(a.row_key, 'UTF8') 
>                AS customer, 
>                Cast(Convert_from(a.data .` l_discount ` , 'double_be') AS FLOAT) 
>                AS availability 
>         FROM   db.tpch_maprdb.lineitem_1 a 
>         WHERE  Convert_from(a.row_key, 'UTF8') = '%004%') AS avail 
>        join 
>               (SELECT Convert_from(b.row_key, 'UTF8') 
>                       AS customer, 
>                Cast( 
>        Convert_from(b.data .` l_discount ` , 'double_be') AS FLOAT) AS 
>                availability 
>         FROM   db.tpch_maprdb.lineitem_1 b 
>         WHERE  Convert_from(b.row_key, 'UTF8') LIKE '%003%') AS prop 
>          ON avail.customer = prop.customer; 
> {code}
> Error:
> {code}
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: NullPointerException
> {code}
> Log attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)