[jira] [Resolved] (HBASE-29696) TableInputFormatBase with NUM_MAPPERS_PER_REGION produces incorrect last InputSplit

["Junegunn Choi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-29696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Junegunn Choi resolved HBASE-29696.
-----------------------------------
    Fix Version/s: 4.0.0-alpha-1
                   2.7.0
                   2.5.16
                   2.6.7
                   3.1.0
                   3.0.1
       Resolution: Fixed

> TableInputFormatBase with NUM_MAPPERS_PER_REGION produces incorrect last InputSplit
> -----------------------------------------------------------------------------------
>
>                 Key: HBASE-29696
>                 URL: https://issues.apache.org/jira/browse/HBASE-29696
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 3.0.0, 2.6.3
>            Reporter: Jasper van Maanen
>            Assignee: mazhengxuan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0-alpha-1, 2.7.0, 2.5.16, 2.6.7, 3.1.0, 3.0.1
>
>
> *How to reproduce*
> Create a table with a single region and add the row key 
> {code:java}
> new byte[] {-1, -1}{code}
> Perform a MapReduce job with at least this setting:
> {code:java}
> "hbase.mapreduce.tableinput.mappers.per.region" = 2 {code}
> The row key is missing in the scan result.
>  
> *Analysis*
> In TableInputF{color:#172b4d}ormatBase#createNInputSplitsUniform there is this code:
> {color}
> {code:java}
> // For special case: startRow or endRow is empty
> if (startRow.length == 0 && endRow.length == 0) {
>   startRow = new byte[1];
>   endRow = new byte[1];
>   startRow[0] = 0;
>   endRow[0] = -1;
> }
> if (startRow.length == 0 && endRow.length != 0) {
>   startRow = new byte[1];
>   startRow[0] = 0;
> }
> if (startRow.length != 0 && endRow.length == 0) {
>   endRow = new byte[startRow.length];
>   for (int k = 0; k < startRow.length; k++) {
>     endRow[k] = -1;
>   }
> } {code}
> Unfortunately, in the first 'if', the endRow is set to 
> {code:java}
> new byte[] {-1} {code}
> But what if there is a row key with
> {code:java}
> new byte[] {-1, -1}{code}
> This row key is after the endRow and will be ignored by the scan. 
> This is also an issue in the third 'if'. Since a row key can be of potentially of an infinite length, setting the end row in the third 'if' also prevents longer row keys (compared to the start row) to be ignored. 
> Therefore, in both situations, the end row should stay empty. I can imagine the endRow is set for the next step:
> {code:java}
> // Split Region into n chunks evenly
> byte[][] splitKeys = Bytes.split(startRow, endRow, true, n - 1); {code}
> In that case, some compensation should be done later to return the correct input splits. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)