You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefan Podkowinski (JIRA)" <ji...@apache.org> on 2017/01/25 12:24:26 UTC
[jira] [Comment Edited] (CASSANDRA-10689)
java.lang.OutOfMemoryError: Direct buffer memory
[ https://issues.apache.org/jira/browse/CASSANDRA-10689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837642#comment-15837642 ]
Stefan Podkowinski edited comment on CASSANDRA-10689 at 1/25/17 12:24 PM:
--------------------------------------------------------------------------
If I understand correctly, the setting will reduce the potential size of allocated native memory per thread for non-direct buffers, but the total allocation size should still be capped by the constant number of pooled threads we use, multiplied by the max. allocated non-direct ByteBuffer size by each thread. What I'm wondering here is where are we using non-direct buffers and if large buffer allocations that would be affected by the setting, are intentional and shouldn't we be able to use smaller buffers instead?
Although the maxCachedBufferSize setting could help avoiding OOM errors, it sounds more like a band-aid to me, for a situation where increased native memory pressure is taking place caused by a memory leak somewhere else, e.g. in the Netty SSL handler (CASSANDRA-13114). Once memory has been depleted, an OOME could as well be raise during mentioned non-direct buffer allocations, even of moderate sizes.
was (Author: spodxx@gmail.com):
If I understand correctly, the setting will reduce the potential size of allocated native memory per thread for non-direct buffers, but the total allocation size should still be capped by the constant number of pooled threads we use, multiplied by the max. allocated non-direct ByteBuffer size by each thread. What I'm wondering here is where are we using non-direct buffers and if large buffer allocations that would be affected by the setting, are intentional and shouldn't we be able to use smaller buffers instead?
Although the maxCachedBufferSize setting could help avoiding OOM errors, it sounds more like a band-aid to me, for a situation where increased native memory pressure is taking place caused by a memory leak somewhere else, e.g. in Netty (CASSANDRA-13114). Once memory has been depleted, an OOME could as well be raise during mentioned non-direct buffer allocations, even of moderate sizes.
> java.lang.OutOfMemoryError: Direct buffer memory
> ------------------------------------------------
>
> Key: CASSANDRA-10689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10689
> Project: Cassandra
> Issue Type: Bug
> Reporter: mlowicki
>
> {code}
> ERROR [SharedPool-Worker-63] 2015-11-11 17:53:16,161 JVMStabilityInspector.java:117 - JVM state determined to be unstable. Exiting forcefully due to:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_80]
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) ~[na:1.7.0_80]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) ~[na:1.7.0_80]
> at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174) ~[na:1.7.0_80]
> at sun.nio.ch.IOUtil.read(IOUtil.java:195) ~[na:1.7.0_80]
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:149) ~[na:1.7.0_80]
> at org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:104) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:310) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:64) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1894) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.columniterator.IndexedSliceReader.setToRowStart(IndexedSliceReader.java:107) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.columniterator.IndexedSliceReader.<init>(IndexedSliceReader.java:83) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:42) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:246) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:270) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1994) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1837) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_80]
> at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164) ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-2.1.11.jar:2.1.11]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)