You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (Created) (JIRA)" <ji...@apache.org> on 2011/12/18 01:11:30 UTC

[jira] [Created] (HBASE-5061) HFileLocalityChecker

HFileLocalityChecker
--------------------

                 Key: HBASE-5061
                 URL: https://issues.apache.org/jira/browse/HBASE-5061
             Project: HBase
          Issue Type: New Feature
            Reporter: Andrew Purtell
            Assignee: Andrew Purtell
            Priority: Minor


org.apache.hadoop.hbase.HFileLocalityChecker [options]

A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.

Where options are:

|-f <file>|Analyze a store file|
|-r <region>|Analyze all store files for the region|
|-t <table>|Analyze all store files for regions of the table served by the local regionserver|
|-h <host>|Consider <host> local, defaults to the local host|
|-v|Verbose operation|


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Affects Version/s: 0.94.1
                       0.96.0
               Status: Patch Available  (was: Open)
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) HFileLocalityChecker

Posted by "Andrew Purtell (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Attachment: StoreFileLocalityChecker.java
    
> HFileLocalityChecker
> --------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401612#comment-13401612 ] 

Hadoop QA commented on HBASE-5061:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12533510/HBASE-5061-0.94.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop2.0.  The patch compiles against the hadoop 2.0 profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//console

This message is automatically generated.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408522#comment-13408522 ] 

Hadoop QA commented on HBASE-5061:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12535469/HBASE-5061-0.94.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop2.0.  The patch compiles against the hadoop 2.0 profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    -1 javac.  The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

    -1 findbugs.  The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.replication.TestReplication
                  org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
                  org.apache.hadoop.hbase.master.TestSplitLogManager

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2338//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2338//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2338//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2338//console

This message is automatically generated.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176322#comment-13176322 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

bq. From looking at the code, this would produce data locality with respect to certain host (if I understand this correctly).

Yes this is the intent, to report locality to a given regionserver host, by default the local host as determined by a HBase style reverse lookup, assuming ops would run it either on an ops box (or the master) and supply the desired "local" host name via the '-h' option, or local to the RS.

I was thinking one use case could be to iterate over each cluster node and trigger major compactions for region(s) depending on the output of this tool. 

Of you are looking for a whole cluster report?

                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Attachment:     (was: StoreFileLocalityChecker.java)
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) HFileLocalityChecker

Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176090#comment-13176090 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

Usage:
{noformat}
./bin/hbase org.apache.hadoop.hbase.util.StoreFileLocalityChecker 
usage: org.apache.hadoop.hbase.util.StoreFileLocalityChecker [-f <arg>]
       [-h <arg>] [-r <arg>] [-s] [-t <arg>] [-v]
 -f,--file <arg>     Analyze a single file. Give the full path.
 -h,--host <arg>     Consider given host as local, defaults to the local
                     host
 -r,--region <arg>   Analyze all store files for the region. Can be the
                     full region name or a partial prefix.
 -s,--summary        Summary only
 -t,--table <arg>    Analyze all store files for regions of the table
                     served by the local regionserver. Give the table
                     name.
 -v,--verbose        Verbose output
{noformat}

{{./bin/hbase org.apache.hadoop.hbase.util.StoreFileLocalityChecker -t TestTable}} produces this output:
{noformat}
hdfs://acer.localdomain:8020/hbase/TestTable/9020c78f14084cfbed55106910a0c175/info/d42cdf9b15e942b1aba8c5752685a29e: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/9020c78f14084cfbed55106910a0c175/info/f4459700138545d4bce236290db606fc: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/ea9d52f79847c606b4b62056a7d5c6d8/info/463c359b8d6040bd9c8b64948611182d: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/b13c50907b32c0aac503d593ef87f586/info/4cde1b33b93a4bfd8f950a64189e7808: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/9d97ffcf9ceac643ba2c3e2b6ad2c416/info/07ec8228b74f434f852aac208437f6e6: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/9d97ffcf9ceac643ba2c3e2b6ad2c416/info/59414c7c8a2a4dd5a0c304eb496f8c78: 1.0
hdfs://acer.localdomain:8020/hbase/TestTable/9d97ffcf9ceac643ba2c3e2b6ad2c416/info/67dd2f83803e4b588926da299c51dffa: 1.0
Block locality index
  Avg: 1.0
  Min: 1.0
  Max: 1.0
{noformat}

{{./bin/hbase org.apache.hadoop.hbase.util.StoreFileLocalityChecker -r TestTable,0000775744}} produces this output:
{noformat}
hdfs://acer.localdomain:8020/hbase/TestTable/b13c50907b32c0aac503d593ef87f586/info/4cde1b33b93a4bfd8f950a64189e7808: 1.0
Block locality index
  Avg: 1.0
  Min: 1.0
  Max: 1.0
{noformat}

{{./bin/hbase org.apache.hadoop.hbase.util.StoreFileLocalityChecker -f hdfs://acer.localdomain:8020/hbase/TestTable/b13c50907b32c0aac503d593ef87f586/info/4cde1b33b93a4bfd8f950a64189e7808}} produces this output:
{noformat}
hdfs://acer.localdomain:8020/hbase/TestTable/b13c50907b32c0aac503d593ef87f586/info/4cde1b33b93a4bfd8f950a64189e7808: 1.0
Block locality index
  Avg: 1.0
  Min: 1.0
  Max: 1.0
{noformat}

Happy to change this in ways people need.
                
> HFileLocalityChecker
> --------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408443#comment-13408443 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

Thanks for the review Ted.

bq. Should @InterfaceAudience.Private be added to the class ?

Ok

bq. Is there other directory that should be skipped ?

Not previously but this is pending live cluster testing. 

bq. Since total isn't included in the report, maybe there is a better name for the above method ?

Ok, let's call it {{reportStats}}

bq. If serverName is a prefix of value, the region would be included in the collection returned. Is that intended ?

Yes, so short names can be used in place of FQDN. 

bq. The above class can be private, right ?

Sure.

                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176326#comment-13176326 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

So, currently this tool is meant to check the locality of files or regions to a given single host, however it could be modified such that if the '-h' option is supplied then the report looks only at the given host, otherwise it will enumerate the live hosts in ClusterStatus and return results for all. While at it, a new option '-j' for reporting in JSON.

Is this more what you have in mind Lars?
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176091#comment-13176091 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

@Weidong, yes this uses the utility functions done for HBASE-4114.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408379#comment-13408379 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

bq. The o.a.h.h.master.balance.RegionLocationFinder does something like this.  It would be cool if functionality could be merged.  Maybe after this goes in I'll open a ticket to do that work

Sounds good to me.

                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Summary: StoreFileLocalityChecker  (was: HFileLocalityChecker)
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176332#comment-13176332 ] 

Lars Hofhansl commented on HBASE-5061:
--------------------------------------

That's what I had envisioned for the -t option at least.
Would be nice to have a report on all regions of a table.
But maybe that would not be useful as the number of region servers grows...?

+1 on JSON output.

                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408410#comment-13408410 ] 

Zhihong Ted Yu commented on HBASE-5061:
---------------------------------------

Nice work.
{code}
+public class StoreFileLocalityChecker {
{code}
Should @InterfaceAudience.Private be added to the class ?
{code}
+    void reportTotals(double min, double max, double average) throws IOException;
{code}
Since total isn't included in the report, maybe there is a better name for the above method ?
{code}
+  private Pair<Path,HDFSBlocksDistribution> getBlocksForPath(Path path) 
{code}
Space after comma.
{code}
+      if (familyDir.getPath().toString().endsWith(".tmp")) {
+        continue;
{code}
Is there other directory that should be skipped ?
{code}
+        // Is the region assigned to the desired server?
+        byte[] value = row.getValue(HConstants.CATALOG_FAMILY, HConstants.SERVER_QUALIFIER);
+        if (value == null || value.length < serverName.length) {
+          return true;
+        }
{code}
If serverName is a prefix of value, the region would be included in the collection returned. Is that intended ?
{code}
+  static class SimpleReporter implements Reporter {
{code}
The above class can be private, right ?
Same with JSONReporter class.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) HFileLocalityChecker

Posted by "Weidong Bian (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174662#comment-13174662 ] 

Weidong Bian commented on HBASE-5061:
-------------------------------------

Hi Andrew, As stack mentioned, this is somewhat similar to HBASE-4114
                
> HFileLocalityChecker
> --------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) HFileLocalityChecker

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171715#comment-13171715 ] 

Lars Hofhansl commented on HBASE-5061:
--------------------------------------

Cool idea. +1
                
> HFileLocalityChecker
> --------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Elliott Clark (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408314#comment-13408314 ] 

Elliott Clark commented on HBASE-5061:
--------------------------------------

The o.a.h.h.master.balance.RegionLocationFinder does something like this.  It would be cool if functionality could be merged.  Maybe after this goes in I'll open a ticket to do that work, since it looks like this is more complete than the balancer version.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Attachment: HBASE-5061-0.94.patch
                HBASE-5061.patch

Latest patch adds a mode where if processing a table (-t) and a host is not specified (-h), then all active hosts in ClusterStatus will be enumerated to process regions for the given table.

This addresses above feedback. Will test on a live cluster and commit if the results are good.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Attachment: HBASE-5061-0.94.patch
                HBASE-5061.patch
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12533510/HBASE-5061-0.94.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop2.0.  The patch compiles against the hadoop 2.0 profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2262//console

This message is automatically generated.)
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-5061:
----------------------------------

    Attachment: HBASE-5061-0.94.patch
                HBASE-5061.patch
    
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408559#comment-13408559 ] 

Hadoop QA commented on HBASE-5061:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12535469/HBASE-5061-0.94.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop2.0.  The patch compiles against the hadoop 2.0 profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    -1 javac.  The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

    -1 findbugs.  The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
     

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2340//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2340//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2340//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2340//console

This message is automatically generated.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061-0.94.patch, HBASE-5061.patch, HBASE-5061.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401571#comment-13401571 ] 

Andrew Purtell commented on HBASE-5061:
---------------------------------------

Updated patch adds '-j' option to produce JSON output.
                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0, 0.94.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-5061-0.94.patch, HBASE-5061.patch
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5061) StoreFileLocalityChecker

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176311#comment-13176311 ] 

Lars Hofhansl commented on HBASE-5061:
--------------------------------------

>From looking at the code, this would produce data locality with respect to certain host (if I understand this correctly).
Wouldn't we want to report locality from the viewpoint the respective regionserver(s)?

I.e. regions for a table might be on many regionservers, but for each the data might be local to the regionserver.

                
> StoreFileLocalityChecker
> ------------------------
>
>                 Key: HBASE-5061
>                 URL: https://issues.apache.org/jira/browse/HBASE-5061
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>         Attachments: StoreFileLocalityChecker.java
>
>
> org.apache.hadoop.hbase.HFileLocalityChecker [options]
> A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage.
> Where options are:
> |-f <file>|Analyze a store file|
> |-r <region>|Analyze all store files for the region|
> |-t <table>|Analyze all store files for regions of the table served by the local regionserver|
> |-h <host>|Consider <host> local, defaults to the local host|
> |-v|Verbose operation|

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira