You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2014/09/05 07:36:26 UTC
[jira] [Commented] (HBASE-11902) RegionServer was blocked while
aborting
[ https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122467#comment-14122467 ]
stack commented on HBASE-11902:
-------------------------------
You mean here:
{code}
"regionserver60020" prio=10 tid=0x00007f85011ca800 nid=0x74d0 in Object.wait() [0x000000004405f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at org.apache.hadoop.hbase.util.DrainBarrier.stopAndDrainOps(DrainBarrier.java:115)
- locked <0x00000002bb325248> (a org.apache.hadoop.hbase.util.DrainBarrier)
at org.apache.hadoop.hbase.util.DrainBarrier.stopAndDrainOps(DrainBarrier.java:85)
at org.apache.hadoop.hbase.regionserver.wal.FSHLog.close(FSHLog.java:923)
at org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:1208)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
at java.lang.Thread.run(Thread.java:744)
{code}
Doesn't seem to be an HDFS issue, just waiting on flushes to complete. You see issues flushing Victor (I've not looked at log).
> RegionServer was blocked while aborting
> ---------------------------------------
>
> Key: HBASE-11902
> URL: https://issues.apache.org/jira/browse/HBASE-11902
> Project: HBase
> Issue Type: Bug
> Components: regionserver, wal
> Affects Versions: 0.98.4
> Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
> Reporter: Victor Xu
> Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log, jstack_hadoop461.cm6.log
>
>
> Generally, regionserver automatically aborts when isHealth() returns false. But it sometimes got blocked while aborting. I saved the jstack and logs, and found out that it was caused by datanodes failures. The "regionserver60020" thread was blocked while closing WAL.
> This issue doesn't happen so frequently, but if it happens, it always leads to huge amount of requests failure. The only way to do is KILL -9.
> I think it's a bug, but I haven't found a decent solution. Does anyone have the same problem?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)