[jira] [Reopened] (HBASE-17302) The region flush request disappeared from flushQueue

["Anoop Sam John (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Anoop Sam John reopened HBASE-17302:
------------------------------------

> The region flush request disappeared from flushQueue
> ----------------------------------------------------
>
>                 Key: HBASE-17302
>                 URL: https://issues.apache.org/jira/browse/HBASE-17302
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 0.98.23, 1.2.4
>            Reporter: Guangxu Cheng
>            Assignee: Guangxu Cheng
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-17302-branch-1-addendum-v1.patch, HBASE-17302-branch-1-addendum.patch, HBASE-17302-branch-1.2-v1.patch, HBASE-17302-branch-master-v1.patch, HBASE-17302-master-addendum-v1.patch, HBASE-17302-master-addendum.patch
>
>
> Region has too many store files delaying flush up to blockingWaitTime ms, and the region flush request is requeued into the flushQueue.
> When the region flush request is requeued into the flushQueue frequently, the request is inexplicably disappeared sometimes. 
> But regionsInQueue still contains the information of the region request, which leads to new flush request can not be inserted into the flushQueue.
> Then, the region will not do flush anymore.
> In order to locate the problem, I added a lot of log in the code.
> {code:title=MemStoreFlusher.java|borderStyle=solid}
> private boolean flushRegion(final HRegion region, final boolean emergencyFlush) {
> long startTime = 0;
> synchronized (this.regionsInQueue) {
>   FlushRegionEntry fqe = this.regionsInQueue.remove(region);
>   // Use the start time of the FlushRegionEntry if available
>   if (fqe != null) {
> 	startTime = fqe.createTime;
>   }
>   if (fqe != null && emergencyFlush) {
> 	// Need to remove from region from delay queue.  When NOT an
> 	// emergencyFlush, then item was removed via a flushQueue.poll.
> 	flushQueue.remove(fqe);
>  }
> }
> {code}
> When encountered emergencyFlush, the region flusher will be removed from the flushQueue.
> By comparing the flushQueue content before and after remove, RegionA should have been removed, it is possible to remove RegionB.
> {code:title=MemStoreFlusher.java|borderStyle=solid}
> public boolean equals(Object obj) {
>   if (this == obj) {
> 	return true;
>   }
>   if (obj == null || getClass() != obj.getClass()) {
> 	return false;
>   }
>   Delayed other = (Delayed) obj;
>   return compareTo(other) == 0;
> }
> {code}
> FlushRegionEntry in achieving the equals function, only comparison of the delay time, if different regions of the same delay time, it is possible that A wrong B.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)