You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2011/04/04 20:35:05 UTC
[jira] [Resolved] (HBASE-3694) high multiput latency due to
checking global mem store size in a synchronized function
[ https://issues.apache.org/jira/browse/HBASE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-3694.
--------------------------
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: [Reviewed]
Committed to TRUNK. Thanks for the nice patch Liyin (and to all who reviewed).
> high multiput latency due to checking global mem store size in a synchronized function
> --------------------------------------------------------------------------------------
>
> Key: HBASE-3694
> URL: https://issues.apache.org/jira/browse/HBASE-3694
> Project: HBase
> Issue Type: Improvement
> Reporter: Liyin Tang
> Assignee: Liyin Tang
> Fix For: 0.92.0
>
> Attachments: 3694-cliffs-counter.txt, Hbase-3694[r1085306], Hbase-3694[r1085306]_2.patch, Hbase-3694[r1085306]_3.patch, Hbase-3694[r1085508]_4.patch, Hbase-3694[r1085592]_7.patch, Hbase-3694[r1085593]_5.patch, Hbase-3694[r1085593]_6.patch
>
>
> The problem is we found the multiput latency is very high.
> In our case, we have almost 22 Regions in each RS and there are no flush happened during these puts.
> After investigation, we believe that the root cause is the function getGlobalMemStoreSize, which is to check the high water mark of mem store.
> This function takes almost 40% of total execution time of multiput when instrumenting some metrics in the code.
> The actual percentage may be more higher. The execution time is spent on synchronize contention.
> One solution is to keep a static var in HRegion to keep the global MemStore size instead of calculating them every time.
> Why using static variable?
> Since all the HRegion objects in the same JVM share the same memory heap, they need to share fate as well.
> The static variable, globalMemStroeSize, naturally shows the total mem usage in this shared memory heap for this JVM.
> If multiple RS need to run in the same JVM, they still need only one globalMemStroeSize.
> If multiple RS run on different JVMs, everything is fine.
> After changing, in our cases, the avg multiput latency decrease from 60ms to 10ms.
> I will submit a patch based on the current trunk.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira