You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2009/01/02 02:39:44 UTC
[jira] Created: (HBASE-1107) NPE in HStoreScanner.updateReaders
NPE in HStoreScanner.updateReaders
----------------------------------
Key: HBASE-1107
URL: https://issues.apache.org/jira/browse/HBASE-1107
Project: Hadoop HBase
Issue Type: Bug
Reporter: Andrew Purtell
Priority: Critical
Fix For: 0.19.0
2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Attachment: (was: 1107-1.patch)
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Attachment: 1107-1.patch
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell reassigned HBASE-1107:
-------------------------------------
Assignee: Andrew Purtell
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667050#action_12667050 ]
stack commented on HBASE-1107:
------------------------------
Thats crazy! I must be digging the hole in the wrong spot?
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.1
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682084#action_12682084 ]
stack commented on HBASE-1107:
------------------------------
The code in updateReaders was changed significantly for 0.19.1 (We don't open readers we already had open).
The NPE above as Andrew has already pointed out comes from here:
{code}
// I think its safe getting key from mem at this stage -- it shouldn't have
// been flushed yet
this.scanners[HSFS_INDEX] = new StoreFileScanner(this.store,
this.timestamp, this. targetCols, this.keys[MEMS_INDEX].getRow());
{code}
It looks like its the this.keys[MEMS_INDEX] is null.
In testing while doing the 0.19.1 changes I was able to make this NPE. It happened when updateReaders was called after the both memory and the file scanners had been exhausted but the scanner had not yet been closed down fully. The attempt to get from the memory key was failing because it had been set to null after its last next.
Now, we keep the last next key and use that instead of trying to go to memcache (thinking on it, memcache exhaustion can always happen ahead of store file scanner exhaustion).
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660907#action_12660907 ]
Andrew Purtell commented on HBASE-1107:
---------------------------------------
Concur that the scanner is closed when the observer gets it.
+1
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-1107) NPE in
HStoreScanner.updateReaders
Posted by "Dave Latham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681940#action_12681940 ]
Dave Latham edited comment on HBASE-1107 at 3/13/09 4:44 PM:
-------------------------------------------------------------
This problem has been taking down regionservers regularly for me as well. Running hbase 0.19.0 on hadoop 0.19.0 with 4 slaves. It's becoming a major stability issue for us.
2009-03-13 16:30:47,728 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: sessions,^@^@^@^@^@^@^Cd^@^@^A^_�x�q^@.IPHONEc8532c72874b66d1165c2f62ed91ef353eb9,1235185327583
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:330)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 3 more
was (Author: davelatham):
This problem has been taking down regionservers regularly for me as well. Running hbase 0.19.0 on hadoop 0.19.0 with 4 slaves.
2009-03-13 16:30:47,728 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: sessions,^@^@^@^@^@^@^Cd^@^@^A^_�x�q^@.IPHONEc8532c72874b66d1165c2f62ed91ef353eb9,1235185327583
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:330)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 3 more
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660902#action_12660902 ]
stack commented on HBASE-1107:
------------------------------
I think this NPE is a symptom of our making the list of changed readers observers sloppier using CopyOnWrite. I think the scanner has been closed. The first thing done on close is that we remove ourselves from the list of changed readers observers but my guess is that there is an outstanding iteration going on. It still has a reference and invoked the observer code AFTER the clearing of the memcache scanner.
I'm not sure your attached patch safe enough Andrew. My worry is that between your new check on #320, this.keys[MEMS_INDEX] could be cleared before its use on #327 by another thread. What you think?
The observer code -- updateReaders method -- shouldn't run if close has been invoked methinks. I was going to add in a 'close' flag. Seems like overkill but ain't sure what else to use as 'close' indicator:
{code}
Index: src/java/org/apache/hadoop/hbase/regionserver/HStoreScanner.java
===================================================================
--- src/java/org/apache/hadoop/hbase/regionserver/HStoreScanner.java (revision 731185)
+++ src/java/org/apache/hadoop/hbase/regionserver/HStoreScanner.java (working copy)
@@ -26,6 +26,7 @@
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
+import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.commons.logging.Log;
@@ -60,6 +61,8 @@
// Used around transition from no storefile to the first.
private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
+ private final AtomicBoolean closing = new AtomicBoolean(false);
+
/** Create an Scanner with a handle on the memcache and HStore files. */
@SuppressWarnings("unchecked")
HStoreScanner(HStore store, byte [][] targetCols, byte [] firstRow,
@@ -294,6 +297,7 @@
}
public void close() {
+ this.closing.set(true);
this.store.deleteChangedReaderObserver(this);
doClose();
}
@@ -309,6 +313,9 @@
// Implementation of ChangedReadersObserver
public void updateReaders() throws IOException {
+ if (this.closing.get()) {
+ return;
+ }
this.lock.writeLock().lock();
try {
MapFile.Reader [] readers = this.store.getReaders();
{code}
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell resolved HBASE-1107.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.19.1
Marking as resolved in 0.19.1
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Fix For: 0.19.1
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Attachment: 1107-1.patch
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Attachment: 1107-1.patch
Attached patch 1107-1 addresses the symptom. Is it enough?
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Fix Version/s: (was: 0.19.0)
0.19.1
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.1
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell reopened HBASE-1107:
-----------------------------------
Assignee: (was: Andrew Purtell)
Seen again, on 0.19.0 release:
2009-01-25 03:56:39,324 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: urls,http|img3.megavideo.com|80|a|8|8b3035357a222073d30d09f2c4ae85.jpg,1232408450783
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushSomeRegions(MemcacheFlusher.java:291)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.reclaimMemcacheMemory(MemcacheFlusher.java:261)
at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdates(HRegionServer.java:1614)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:895)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:330)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 10 more
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.1
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683671#action_12683671 ]
stack commented on HBASE-1107:
------------------------------
Yeah, should be fixed by 0.19.1 by my reckoning.
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Status: Patch Available (was: Open)
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-1107) NPE in
HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660907#action_12660907 ]
apurtell edited comment on HBASE-1107 at 1/5/09 12:35 PM:
----------------------------------------------------------------
Concur that the scanner is closed when the observer gets it.
+1
I'll commit.
was (Author: apurtell):
Concur that the scanner is closed when the observer gets it.
+1
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Attachment: (was: 1107-1.patch)
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell reassigned HBASE-1107:
-------------------------------------
Assignee: Andrew Purtell
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed. Passes all local tests.
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660363#action_12660363 ]
Andrew Purtell commented on HBASE-1107:
---------------------------------------
Right above line 322 in HStoreScanner.java is this comment:
\\
{code}
// I think it is safe getting key from mem at this stage -- it shouldn't have
// been flushed yet
{code}
So I think this is no longer true.
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Critical
> Fix For: 0.19.0
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1107:
----------------------------------
Priority: Blocker (was: Critical)
Affects Version/s: 0.20.0
0.19.1
Fix Version/s: (was: 0.19.1)
Got another of these today.
2009-02-09 00:16:55,081 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: content,26ac2c3fd24ac7ba1032e7b68973b9fa,1233995252171
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushSomeRegions(MemcacheFlusher.java:291)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.reclaimMemcacheMemory(MemcacheFlusher.java:261)
at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdates(HRegionServer.java:1619)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:895)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:330)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 10 more
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Dave Latham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681940#action_12681940 ]
Dave Latham commented on HBASE-1107:
------------------------------------
This problem has been taking down regionservers regularly for me as well. Running hbase 0.19.0 on hadoop 0.19.0 with 4 slaves.
2009-03-13 16:30:47,728 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: sessions,^@^@^@^@^@^@^Cd^@^@^A^_�x�q^@.IPHONEc8532c72874b66d1165c2f62ed91ef353eb9,1235185327583
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:330)
at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 3 more
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1107:
-------------------------
Comment: was deleted
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683653#action_12683653 ]
Andrew Purtell commented on HBASE-1107:
---------------------------------------
So head of 0.19 branch includes a resolution for this?
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1107) NPE in HStoreScanner.updateReaders
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671929#action_12671929 ]
stack commented on HBASE-1107:
------------------------------
Do you think then Andrew the case is that we are moving from no scanners against files to our first one as the comment suggests? ("Presume that we went from no readers to at least one -- need to pu....").
If so, do you think this.keys and this.scanners need resizing to accomodate the new file scanner?
If so, should we make a new method that does this and have the constructor call it setting them up initially and also this bit of code where we update the readers, have it call it too? The new method should be synchronized since these are data members?
Please allow that the above speculation may be off. My line numbers do not align with those reported above so I may not have the story right.
> NPE in HStoreScanner.updateReaders
> ----------------------------------
>
> Key: HBASE-1107
> URL: https://issues.apache.org/jira/browse/HBASE-1107
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.1, 0.20.0
> Reporter: Andrew Purtell
> Priority: Blocker
> Attachments: 1107-1.patch
>
>
> 2009-01-01 23:55:41,629 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: content,cff13605e2ea6ce0b221ac864687bf08,1230777531253
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:880)
> at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:773)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HStoreScanner.updateReaders(HStoreScanner.java:322)
> at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:865)
> ... 3 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.