You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2019/02/25 03:43:15 UTC

[GitHub] ly923976094 opened a new issue #3874: consumer out of memory 

ly923976094 opened a new issue #3874: consumer out of memory 
URL: https://github.com/apache/incubator-pinot/issues/3874
 
 
   When the consumer is consuming, when the state is transitioning, memory out of memory, consumption stops. What's going on here
   server configuration（6+32G, jvm16G）
   java.lang.reflect.InvocationTargetException
           at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:430)
           at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:344)
           at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:96)
           at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:47)
           at java.util.concurrent.FutureTask.run(FutureTask.java:266)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           at java.lang.Thread.run(Thread.java:745)
   Caused by: java.lang.OutOfMemoryError: Direct buffer memory
           at java.nio.Bits.reserveMemory(Bits.java:693)
           at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
           at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
           at com.linkedin.pinot.core.segment.memory.PinotByteBuffer.allocateDirect(PinotByteBuffer.java:36)
           at com.linkedin.pinot.core.segment.memory.PinotByteBuffer.loadFile(PinotByteBuffer.java:40)
           at com.linkedin.pinot.core.segment.memory.PinotDataBuffer.loadFile(PinotDataBuffer.java:144)
           at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.mapAndSliceFile(SingleFileIndexDirectory.java:284)
           at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.mapBufferEntries(SingleFileIndexDirectory.java:265)
           at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.load(SingleFileIndexDirectory.java:190)
           at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.<init>(SingleFileIndexDirectory.java:86)
           at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.loadData(SegmentLocalFSDirectory.java:198)
           at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.load(SegmentLocalFSDirectory.java:179)
           at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.<init>(SegmentLocalFSDirectory.java:77)
           at com.linkedin.pinot.core.segment.store.SegmentDirectory.createFromLocalFS(SegmentDirectory.java:108)
           at com.linkedin.pinot.core.indexsegment.immutable.ImmutableSegmentLoader.load(ImmutableSegmentLoader.java:108)
           at com.linkedin.pinot.core.data.manager.realtime.RealtimeTableDataManager.replaceLLSegment(RealtimeTableDataManager.java:291)
           at com.linkedin.pinot.core.data.manager.realtime.RealtimeTableDataManager.downloadAndReplaceSegment(RealtimeTableDataManager.java:279)
           at com.linkedin.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.downloadSegmentAndReplace(LLRealtimeSegmentDataManager.java:945)
           at com.linkedin.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.goOnlineFromConsuming(LLRealtimeSegmentDataManager.java:934)
           at com.linkedin.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromConsuming(SegmentOnlineOfflineStateModelFactory.java:118)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org