You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ian Friedman (JIRA)" <ji...@apache.org> on 2015/10/13 20:15:06 UTC
[jira] [Updated] (HBASE-14598) ByteBufferOutputStream grows its
HeapByteBuffer beyond JVM limitations
[ https://issues.apache.org/jira/browse/HBASE-14598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ian Friedman updated HBASE-14598:
---------------------------------
Description:
We noticed that in returning a Scan against a region containing particularly large (wide) rows that it is possible during ByteBufferOutputStream. checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM allows (which I believe is Integer.MAX_VALUE), which then throws a OutOfMemoryError. This lead to us dealing with cascading region server death as the RegionServer hosting the region died, opened on a new server, the client retried the scan, and the new RS died as well.
I believe ByteBufferOutputStream should not try to create ByteBuffers that large and instead throw an exception back up. The limit should probably be something like Integer.MAX_VALUE-8, as that is what ArrayList uses. ref: http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/ArrayList.java#221
was:
We noticed that in returning a Scan against a region containing particularly large (wide) rows that it is possible during ByteBufferOutputStream. checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM allows (which I believe is Integer.MAX_VALUE), which then throws a OutOfMemoryError. This lead to us dealing with cascading region server death as the RegionServer hosting the region died, opened on a new server, the client retried the scan, and the new RS died as well.
I believe ByteBufferOutputStream should not try to create ByteBuffers that large and instead throw an exception back up.
> ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations
> ----------------------------------------------------------------------
>
> Key: HBASE-14598
> URL: https://issues.apache.org/jira/browse/HBASE-14598
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.98.12
> Reporter: Ian Friedman
> Assignee: Ian Friedman
>
> We noticed that in returning a Scan against a region containing particularly large (wide) rows that it is possible during ByteBufferOutputStream. checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM allows (which I believe is Integer.MAX_VALUE), which then throws a OutOfMemoryError. This lead to us dealing with cascading region server death as the RegionServer hosting the region died, opened on a new server, the client retried the scan, and the new RS died as well.
> I believe ByteBufferOutputStream should not try to create ByteBuffers that large and instead throw an exception back up. The limit should probably be something like Integer.MAX_VALUE-8, as that is what ArrayList uses. ref: http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/ArrayList.java#221
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)