[jira] Commented: (HBASE-1892) [performance] make hbase splits run faster

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791697#action_12791697 ] 

stack commented on HBASE-1892:
------------------------------

Here are some observations by J-D from a recent (offlist) mail:

{code}
So, the way it currently works, you should expect pauses of 6 to 10
seconds every time there's a split. If the system has more load, it
can take a bit longer. This is "how it works" in the current master
architecture... [to be changed in 0.21 hbase -- St.Ack]. This configuration:

 <property>
   <name>hbase.regionserver.msginterval</name>
   <value>3000</value>
   <description>Interval between messages from the RegionServer to HMaster
   in milliseconds.  Default is 3 seconds.
   </description>
 </property>

makes it that every split takes up to 3 second to report to master and then
3 more seconds for the master to assign the new regions. Coupled with
that is the client retries backoff. In HConnectionManager we wait
hbase.client.pause (default 2 secs) times

public static int RETRY_BACKOFF[] = { 1, 1, 1, 2, 2, 4, 4, 8, 16, 32 };

in order. So for a split you wait 2 + 2 + 2 seconds in the client. If
for some reason it took longer to split, you will wait 6+2*2=10
seconds (or more).

{code}

A couple of notes on the above.

Not so long ago I changed the regionserver so that as soon as it had a message, it'd send the master and not wait on the hbase.regionserver.msginterval to elapse.

My guess is that if more than one regionserver, rare would be the case when we would have to wait 3 seconds to send out the assignment of daughters -- my guess is that a regionserver would check in well before then.

But then there is the open of the region and informing master.

Maybe we should add to the RETRY_BACKOFF a bunch more 1s at the start... and maybe down the hbase.client.pause default from 2 seconds to 1.

> [performance] make hbase splits run faster
> ------------------------------------------
>
>                 Key: HBASE-1892
>                 URL: https://issues.apache.org/jira/browse/HBASE-1892
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: stack
>             Fix For: 0.21.0
>
>
> hbase-1506 tried and failed making splits faster in 0.20 context.  This issue is about doing it in 0.21 where we'll have to tools to do.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.