You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Simon Dircks (Created) (JIRA)" <ji...@apache.org> on 2012/01/26 19:27:41 UTC
[jira] [Created] (HBASE-5285) runtime exception -- cached an
already cached block -- during compaction
runtime exception -- cached an already cached block -- during compaction
------------------------------------------------------------------------
Key: HBASE-5285
URL: https://issues.apache.org/jira/browse/HBASE-5285
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.92.0
Environment: hadoop-1.0 and hbase-0.92
18 node cluster, dedicated namenode, zookeeper, hbasemaster, and YCSB client machine.
latest YCSB
Reporter: Simon Dircks
Priority: Trivial
#On YCSB client machine:
/usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p recordcount=5000000 -s > load.dat
loaded 5mil records, that created 8 regions. (balanced all onto the same RS)
/usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p operationcount=5000000 -threads 10 -s > transaction.dat
#On RS that was holding the 8 regions above.
2012-01-25 23:23:51,556 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Successfully transitioned node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
2012-01-25 23:23:51,616 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
java.lang.RuntimeException: Cached an already cached block
at org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:268)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:276)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:487)
at org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:168)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:181)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2861)
at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1432)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1424)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1400)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3688)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3581)
at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1771)
at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1325)
2012-01-25 23:23:51,656 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Attempting to transition node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5285) runtime exception -- cached an
already cached block -- during compaction
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197311#comment-13197311 ]
stack commented on HBASE-5285:
------------------------------
Can you make this happen reliably Simon?
> runtime exception -- cached an already cached block -- during compaction
> ------------------------------------------------------------------------
>
> Key: HBASE-5285
> URL: https://issues.apache.org/jira/browse/HBASE-5285
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.92.0
> Environment: hadoop-1.0 and hbase-0.92
> 18 node cluster, dedicated namenode, zookeeper, hbasemaster, and YCSB client machine.
> latest YCSB
> Reporter: Simon Dircks
> Priority: Trivial
>
> #On YCSB client machine:
> /usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p recordcount=5000000 -s > load.dat
> loaded 5mil records, that created 8 regions. (balanced all onto the same RS)
> /usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p operationcount=5000000 -threads 10 -s > transaction.dat
> #On RS that was holding the 8 regions above.
> 2012-01-25 23:23:51,556 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Successfully transitioned node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
> 2012-01-25 23:23:51,616 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.lang.RuntimeException: Cached an already cached block
> at org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:268)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:276)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:487)
> at org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:168)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:181)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2861)
> at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1432)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1424)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1400)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3688)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3581)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1771)
> at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1325)
> 2012-01-25 23:23:51,656 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Attempting to transition node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5285) runtime exception -- cached an
already cached block -- during compaction
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197349#comment-13197349 ]
Zhihong Yu commented on HBASE-5285:
-----------------------------------
Could this be due to hash code collision of BlockCacheKey ?
> runtime exception -- cached an already cached block -- during compaction
> ------------------------------------------------------------------------
>
> Key: HBASE-5285
> URL: https://issues.apache.org/jira/browse/HBASE-5285
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.92.0
> Environment: hadoop-1.0 and hbase-0.92
> 18 node cluster, dedicated namenode, zookeeper, hbasemaster, and YCSB client machine.
> latest YCSB
> Reporter: Simon Dircks
> Priority: Trivial
>
> #On YCSB client machine:
> /usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p recordcount=5000000 -s > load.dat
> loaded 5mil records, that created 8 regions. (balanced all onto the same RS)
> /usr/local/bin/java -cp "build/ycsb.jar:db/hbase/lib/*:db/hbase/conf/" com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family1 -p operationcount=5000000 -threads 10 -s > transaction.dat
> #On RS that was holding the 8 regions above.
> 2012-01-25 23:23:51,556 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Successfully transitioned node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
> 2012-01-25 23:23:51,616 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.lang.RuntimeException: Cached an already cached block
> at org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:268)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:276)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:487)
> at org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:168)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:181)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2861)
> at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1432)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1424)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1400)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3688)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3581)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1771)
> at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1325)
> 2012-01-25 23:23:51,656 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x134f70a343101a0 Attempting to transition node 162702503c650e551130e5fb588b3ec2 from RS_ZK_REGION_SPLIT to RS_ZK_REGION_SPLIT
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira