You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/12/17 06:48:18 UTC
[jira] Created: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
--------------------------------------------------------------------------------------
Key: HBASE-2052
URL: https://issues.apache.org/jira/browse/HBASE-2052
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack
Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-2052.
---------------------------------------
Resolution: Fixed
Assignee: Jean-Daniel Cryans
Hadoop Flags: [Reviewed]
I added the comment as ask by Stack and committed to branch.
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792085#action_12792085 ]
Jean-Daniel Cryans commented on HBASE-2052:
-------------------------------------------
I also committed to trunk, will be changed when master is redone.
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798181#action_12798181 ]
stack commented on HBASE-2052:
------------------------------
On large clusers, master would be doing more work when interval is low. The interval should be upped on bigger clusters (Usual clusters sizes are 10-20 nodes that we hear of).
Yeah, if a regionserver has something to send the master, we've already made it so that is sent immediately without waiting on the msg interval to expire but, then once the message to split makes it to master, it won't assign until a regionserver checks in (we're adding 'push' of messages in 0.21 using zk watchers... master will add region to assign to a queue up in zk... this will trigger a watcher on designated regionserver).
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2052:
-------------------------
Fix Version/s: 0.20.3
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.20.3
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-2052:
--------------------------------------
Attachment: HBASE-2052.patch
Setting the client pause and msginterval to 1. I don't think we should change the retries backoff. I tried it on a PE and created tables in the shell, I like it.
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797978#action_12797978 ]
dhruba borthakur commented on HBASE-2052:
-----------------------------------------
Does this patch has any performance implication for large clusters?
It appears to me that when a region is split, the RegionServer has to inform the master. This occurs in the next heartbeat from the region server to the master. If so, can the regionserver send a immediate heartbeat message after a region split instead of waiting for the next hbase.regionserver.msginteral to expire?
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792017#action_12792017 ]
stack commented on HBASE-2052:
------------------------------
+1
On commit, I'd add a note that might want to increase interval if cluster is larger (> 10 nodes):
{code}
- in milliseconds. Default is 3 seconds.
+ in milliseconds. Default is 1 seconds.
{code}
Add it above between milliseconds and Default.
Thanks for doing this J-D.
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2052) Make hbase more 'live' when comes to
noticing table creation, splits, etc., for 0.20.3
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798179#action_12798179 ]
Jean-Daniel Cryans commented on HBASE-2052:
-------------------------------------------
Going from 3 to 1 puts more stress on the master for large clusters, but also gives faster splits. I can't give a number on how slower the master can be, or how busier it will be with X number of nodes. I expect that no one will see the difference.
The best would be to have the heartbeat separated from the message passing. This is one of the goal of the master redesign.
> Make hbase more 'live' when comes to noticing table creation, splits, etc., for 0.20.3
> --------------------------------------------------------------------------------------
>
> Key: HBASE-2052
> URL: https://issues.apache.org/jira/browse/HBASE-2052
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.3
>
> Attachments: HBASE-2052.patch
>
>
> Clusters > 10 are less frequent that smaller clusters. Defaults for when messages are sent from regionserver to master would seem to favor clusters that are much larger than ten nodes. This issue is about changing them so the basic hbase setup is more live. Its about downing the interval between RS sending messages from 3 to 1 second and of making clients retry more frequently at least at the start of the timeout period so they pick up changes the quicker. This should help with stuff like noticing splits and new regions coming on line. See HBASE-1892 for background.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.