You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "ryan rawson (JIRA)" <ji...@apache.org> on 2010/01/05 00:14:54 UTC
[jira] Commented: (HBASE-2073) IllegalArgumentException causing
regionserver failure
[ https://issues.apache.org/jira/browse/HBASE-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796424#action_12796424 ]
ryan rawson commented on HBASE-2073:
------------------------------------
the log doesnt have much detail, typically in the past these kinds of errors can indicate short read from HDFS. I'm not seeing any of those in this logfile, is there anything interesting in the HDFS logs?
For this error to have happened, there should have been a short read in the block, which is "impossible".
> IllegalArgumentException causing regionserver failure
> -----------------------------------------------------
>
> Key: HBASE-2073
> URL: https://issues.apache.org/jira/browse/HBASE-2073
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.2
> Environment: Ubuntu 8.10, Java 1.6.0_10, HBase 0.20.2
> Reporter: Greg Lu
> Assignee: ryan rawson
> Fix For: 0.20.3
>
> Attachments: hbase-hadoop-regionserver-factory05.lab.mtl.log
>
>
> After a regionserver went down last night, I checked its logs and found the following exception:
> 2009-12-29 00:17:27,663 INFO org.apache.hadoop.hbase.regionserver.HLog: Roll /hbase/amsterdam_factory/.logs/factory05.lab.mtl,60020,1262042255724/hlog.dat.1262060247637, entries=1830, calcsize=22946017, filesize=22758899. New hlog /hbase/amsterdam_factory/.logs/factory05.lab.mtl,60020,1262042255724/hlog.dat.1262063847659
> 2009-12-29 00:34:36,210 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1114)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:79)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:189)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1776)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1719)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1944)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> 2009-12-29 00:34:36,214 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60020, call next(4170645244799815171, 1) from 192.168.1.108:53401: error: java.io.IOException: java.lang.IllegalArgumentException
> java.io.IOException: java.lang.IllegalArgumentException
> at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:869)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:859)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1965)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1114)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:79)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:189)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1776)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1719)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1944)
> ... 5 more
> Looks like this bug was encountered before at https://issues.apache.org/jira/browse/HBASE-1495 and spanned a few JIRAs. It's supposed to be resolved as of 0.20.0, but we're running 0.20.2 and it took down one of our regionservers.
> I'm also attaching more of the log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.