You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org> on 2010/07/10 01:52:49 UTC

[jira] Created: (HBASE-2828) HTable unnecessarily coupled with HMaster

HTable unnecessarily coupled with HMaster
-----------------------------------------

                 Key: HBASE-2828
                 URL: https://issues.apache.org/jira/browse/HBASE-2828
             Project: HBase
          Issue Type: Bug
          Components: client
    Affects Versions: 0.21.0
            Reporter: Nicolas Spiegelberg
             Fix For: 0.21.0


HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Reopened: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Gray reopened HBASE-2828:
----------------------------------


This is in some 0.89 branches but was left out from 0.90 (I guess during master rewrite commit).

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Gray updated HBASE-2828:
---------------------------------

    Status: Patch Available  (was: Reopened)

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828-0.90.patch, HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack resolved HBASE-2828.
--------------------------

    Hadoop Flags: [Reviewed]
        Assignee: Nicolas Spiegelberg
      Resolution: Fixed

Committed. Thank you for the patch Nicolas. Applied TRUNK.

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Gray updated HBASE-2828:
---------------------------------

    Attachment: HBASE-2828-0.90.patch

Looks like the documentation changes made it, just not the HTable.getCurrentNrHRS().  Patch changes implementation of that method (same principle as original patch but slightly different0.

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828-0.90.patch, HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-2828:
-------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I committed your patch Jon.

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828-0.90.patch, HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Spiegelberg updated HBASE-2828:
---------------------------------------

    Attachment: HBASE-2828.a.patch

Found the API I needed.   zk.exists() returns a Stat class, which contains the children count.

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Spiegelberg updated HBASE-2828:
---------------------------------------

    Attachment: HBASE-2828.patch

I just kept this simple and used existing API calls to grab the zkWrapper, get all RS nodes, and call size().  It would be nice if ZK provided a getChildrenCount(), so it didn't have to list all the paths, but I assumed this shouldn't be a significant performance concern, even for larger clusters.

Note that I also used this opportunity to clean up the class header documentation for HTable & HBaseAdmin.  Feel free to ignore those changes if you have your own doc update.

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928012#action_12928012 ] 

Nicolas Spiegelberg commented on HBASE-2828:
--------------------------------------------

can we add some explicit comment in there about purposefully not going to Master for the RS count.  I wouldn't want a 3rd occurrence of this...

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828-0.90.patch, HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-2828) HTable unnecessarily coupled with HMaster

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928051#action_12928051 ] 

stack commented on HBASE-2828:
------------------------------

done

> HTable unnecessarily coupled with HMaster
> -----------------------------------------
>
>                 Key: HBASE-2828
>                 URL: https://issues.apache.org/jira/browse/HBASE-2828
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-2828-0.90.patch, HBASE-2828.a.patch, HBASE-2828.patch
>
>
> HTable constructor calls "getCurrentNrHRS()" to get the region server count for thread pool creation.  This code calls HBaseAdmin.getClusterStatus() [aka: the HMaster] to get the server count.  This information can be scraped from counting the ZooKeeper /hbase/rs/--- ZNodes.  Need to remove unnecessary master queries when ZooKeeper can do the same job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.