You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/04/16 02:47:57 UTC
[GitHub] [incubator-druid] shizhengchao opened a new issue #7485: why so
many littleEndByteBuff
shizhengchao opened a new issue #7485: why so many littleEndByteBuff
URL: https://github.com/apache/incubator-druid/issues/7485
a large number of "littleEndByteBuff" logļ¼causing OOM
### Affected Version
druid-0.13.0-incubating
### Description
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,049]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,050]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,051]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,052]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,053]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,054]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,055]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,056]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,057]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,058]
2019-04-14T08:15:02,242 INFO [fcbox_send-2019-04-13T00:00:00.000Z-persist-n-merge] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3,059]
Exception in thread "plumber_merge_0" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.druid.segment.CompressedPools$4.get(CompressedPools.java:105)
at org.apache.druid.segment.CompressedPools$4.get(CompressedPools.java:98)
at org.apache.druid.collections.StupidPool.makeObjectWithHandler(StupidPool.java:116)
at org.apache.druid.collections.StupidPool.take(StupidPool.java:107)
at org.apache.druid.segment.CompressedPools.getByteBuf(CompressedPools.java:113)
at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:49)
at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:28)
at org.apache.druid.segment.data.GenericIndexed$BufferIndexed.bufferedIndexedGet(GenericIndexed.java:444)
at org.apache.druid.segment.data.GenericIndexed$2.get(GenericIndexed.java:599)
at org.apache.druid.segment.data.BlockLayoutColumnarLongsSupplier$1.loadBuffer(BlockLayoutColumnarLongsSupplier.java:84)
at org.apache.druid.segment.data.BlockLayoutColumnarLongsSupplier$1.get(BlockLayoutColumnarLongsSupplier.java:73)
at org.apache.druid.segment.data.ColumnarLongs$1HistoricalLongColumnSelector.getLong(ColumnarLongs.java:60)
at org.apache.druid.segment.QueryableIndexIndexableAdapter$RowIteratorImpl.setRowPointerValues(QueryableIndexIndexableAdapter.java:317)
at org.apache.druid.segment.QueryableIndexIndexableAdapter$RowIteratorImpl.moveToNext(QueryableIndexIndexableAdapter.java:298)
at org.apache.druid.segment.ForwardingRowIterator.moveToNext(ForwardingRowIterator.java:62)
at org.apache.druid.segment.ForwardingRowIterator.moveToNext(ForwardingRowIterator.java:62)
at org.apache.druid.segment.MergingRowIterator.lambda$new$0(MergingRowIterator.java:84)
at java.util.stream.IntPipeline$9$1.accept(IntPipeline.java:343)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438)
at org.apache.druid.segment.MergingRowIterator.<init>(MergingRowIterator.java:92)
at org.apache.druid.segment.IndexMergerV9.makeMergedTimeAndDimsIterator(IndexMergerV9.java:1032)
at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:180)
at org.apache.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:915)
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:833)
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:811)
at org.apache.druid.segment.realtime.plumber.RealtimePlumber$2.doRun(RealtimePlumber.java:433)
at org.apache.druid.common.guava.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
As you see, nearly 3,060 "littleEndByteBuf", and my task OOM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org