You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Bo Cui (JIRA)" <ji...@apache.org> on 2019/01/15 08:24:00 UTC
[jira] [Comment Edited] (HBASE-21721) reduce write#syncs() times
[ https://issues.apache.org/jira/browse/HBASE-21721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742819#comment-16742819 ]
Bo Cui edited comment on HBASE-21721 at 1/15/19 8:23 AM:
---------------------------------------------------------
test after modifying FSHLog
if regions num is 200, Requests Per Second = 7000+
and running-log does not print Slow sync frequently
was (Author: bo cui):
test after modifying FSHLog
if regions num is 200, Requests Per Second = 7000+
and running-log does not print Slow sync frequently
but puts is still slow, MemStoreFlusher is not frequent, i suspect this is because of FSHLog
{code:title=rs.jstack|borderStyle=solid}
"RpcServer.FifoWFPBQ.default.handler=1999,queue=199,port=21302" #2160 daemon prio=5 os_prio=0 tid=0x00007fe6169cd000 nid=0x1c99e in Object.wait() [0x00007fe4137fd000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:172)
- locked <0x0000000602266950> (a org.apache.hadoop.hbase.regionserver.wal.SyncFuture)
at org.apache.hadoop.hbase.regionserver.wal.FSHLog.blockOnSync(FSHLog.java:1422)
at org.apache.hadoop.hbase.regionserver.wal.FSHLog.publishSyncThenBlockOnCompletion(FSHLog.java:1416)
at org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:1523)
at org.apache.hadoop.hbase.regionserver.HRegion.syncOrDefer(HRegion.java:8632)
at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3427)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3043)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2985)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:930)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:859)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2480)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35080)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2445)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:191)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
{code}
> reduce write#syncs() times
> --------------------------
>
> Key: HBASE-21721
> URL: https://issues.apache.org/jira/browse/HBASE-21721
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 1.3.1, 2.1.1
> Reporter: Bo Cui
> Priority: Major
>
> the number of write#syncs can be reduced by updating the highestUnsyncedSequence：
> before write#sync(), get the current highestUnsyncedSequence
> after write#sync, highestSyncedSequence=highestUnsyncedSequence
>
> {code:title=FSHLog.java|borderStyle=solid}
> // Some comments here
> public void run()
> {
> long currentSequence;
> while (!isInterrupted()) {
> int syncCount = 0;
> try {
> while (true) {
> ...
> try {
> Trace.addTimelineAnnotation("syncing writer");
> long unSyncedFlushSeq = highestUnsyncedSequence;
> writer.sync();
> Trace.addTimelineAnnotation("writer synced");
> if( unSyncedFlushSeq > currentSequence ) currentSequence = unSyncedFlushSeq;
> currentSequence = updateHighestSyncedSequence(currentSequence);
> } catch (IOException e) {
> LOG.error("Error syncing, request close of WAL", e);
> lastException = e;
> } catch (Exception e) {
> ...
> }
> }
> {code}
> Add code
> long unSyncedFlushSeq = highestUnsyncedSequence;
> if( unSyncedFlushSeq > currentSequence ) currentSequence = unSyncedFlushSeq;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)