You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Sujee Maniyam (Created) (JIRA)" <ji...@apache.org> on 2012/03/10 00:54:57 UTC

[jira] [Created] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

--presplit option creates a first split with rowkey-end=0
---------------------------------------------------------

                 Key: HBASE-5559
                 URL: https://issues.apache.org/jira/browse/HBASE-5559
             Project: HBase
          Issue Type: Bug
          Components: util
            Reporter: Sujee Maniyam
            Assignee: Sujee Maniyam
            Priority: Trivial


HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.

when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.

For example, 
if total keyspace is 100, and splits requested are 5, 
generated splits => [0, 20, 40, 60, 80]
it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Sujee Maniyam (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sujee Maniyam updated HBASE-5559:
---------------------------------

    Status: Patch Available  (was: Open)

HBASE-5559-v2.patch
                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228093#comment-13228093 ] 

Lars Hofhansl commented on HBASE-5559:
--------------------------------------

Shouldn't this rather be [20, 40, 60, 80] to demarcate 5 regions?
The first region holds keys <20, the 2nd [20, 40), and so on, and the last one holds key >= 80.

                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228095#comment-13228095 ] 

Lars Hofhansl commented on HBASE-5559:
--------------------------------------

specifically the key ranges produced should be:
empty, 20
20, 40
40, 60
60, 80
80, empty

                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228818#comment-13228818 ] 

Hadoop QA commented on HBASE-5559:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12518265/HBASE-5559-v2.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 161 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.mapreduce.TestImportTsv
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1178//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1178//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1178//console

This message is automatically generated.
                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Sujee Maniyam (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sujee Maniyam updated HBASE-5559:
---------------------------------

    Status: Open  (was: Patch Available)
    
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228135#comment-13228135 ] 

Hadoop QA commented on HBASE-5559:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12518122/5559_v1.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.regionserver.TestAtomicOperation
                  org.apache.hadoop.hbase.mapreduce.TestImportTsv
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1171//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1171//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1171//console

This message is automatically generated.
                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230665#comment-13230665 ] 

stack commented on HBASE-5559:
------------------------------

Sujee Since you are learning how to contrib., how about your figuring how to add a test for your change.  Your life testing might be easier if you made getSplits static and passed in this.presplitRegions as an argument (you can have getSplits call it).  In your test you'd exercise the different combinations passing a this.presplitRegions with no elements and then with a few different values for R?

Patch is looking good otherwise.
                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230699#comment-13230699 ] 

Lars Hofhansl commented on HBASE-5559:
--------------------------------------

Patch looks good.
(If I wanted to split hairs I'd say that if the number of splits is approaching the size of the keyspace you'd run into funky integer arithmetic.
Say the keyspace is 100 and presplits is 30. You'd get 29 regions of size 3 and one of size 10... But do not want to split hairs :) )

                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Sujee Maniyam (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sujee Maniyam updated HBASE-5559:
---------------------------------

    Attachment: 5559_v1.patch
    
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Sujee Maniyam (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sujee Maniyam updated HBASE-5559:
---------------------------------

    Attachment: HBASE-5559-v2.patch

revised patch
                
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch, HBASE-5559-v2.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

Posted by "Sujee Maniyam (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sujee Maniyam updated HBASE-5559:
---------------------------------

    Release Note: bug fix :  fix for --presplit option creating a split with zero end key (hence not getting any data)
          Status: Patch Available  (was: Open)
    
> --presplit option creates a first split with rowkey-end=0
> ---------------------------------------------------------
>
>                 Key: HBASE-5559
>                 URL: https://issues.apache.org/jira/browse/HBASE-5559
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>            Reporter: Sujee Maniyam
>            Assignee: Sujee Maniyam
>            Priority: Trivial
>              Labels: benchmark
>         Attachments: 5559_v1.patch
>
>
> HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
> when the splits are generated, the first split has row-end-key=0 (zero).  Hence this split doesn't get any data.
> For example, 
> if total keyspace is 100, and splits requested are 5, 
> generated splits => [0, 20, 40, 60, 80]
> it should be => [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira