You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/04/10 01:28:16 UTC
[jira] [Commented] (DRILL-496) drill stuck in loop when query
parquet file and eventually ran out of memory
[ https://issues.apache.org/jira/browse/DRILL-496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964803#comment-13964803 ]
ASF GitHub Bot commented on DRILL-496:
--------------------------------------
GitHub user StevenMPhillips opened a pull request:
https://github.com/apache/incubator-drill/pull/47
DRILL-496 drill stuck in loop when query parquet file and eventually ran...
... out of memory
Added condition in NullableColumnReader to break out of loop when there are no more pages to read.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StevenMPhillips/incubator-drill DRILL-496
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-drill/pull/47.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #47
----
commit fe9891c9c84ffcee3eefdb3fffc936b76519a175
Author: Steven Phillips <sp...@maprtech.com>
Date: 2014-04-09T22:14:49Z
DRILL-496 drill stuck in loop when query parquet file and eventually ran out of memory
Added condition in NullableColumnReader to break out of loop when there are no more pages to read.
----
> drill stuck in loop when query parquet file and eventually ran out of memory
> ----------------------------------------------------------------------------
>
> Key: DRILL-496
> URL: https://issues.apache.org/jira/browse/DRILL-496
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.0.0-milestone-2
> Reporter: Chun Chang
> Assignee: Steven Phillips
> Priority: Blocker
>
> Ran the following Mondrian query against parquet format file and hit the issue that drill bit in infinite loop, and eventually run out of memory.
> 0: jdbc:drill:schema=dfs> select * from days;
> drillbit log:
> Exception in thread "WorkManager-3" java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658)
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> at io.netty.buffer.PoolArenaL$DirectArena.newUnpooledChunk(PoolArenaL.java:387)
> at io.netty.buffer.PoolArenaL.allocateHuge(PoolArenaL.java:152)
> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:129)
> at io.netty.buffer.PoolArenaL.reallocate(PoolArenaL.java:234)
> at io.netty.buffer.PooledByteBufL.capacity(PooledByteBufL.java:116)
> at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:250)
> at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:807)
> at org.apache.drill.exec.memory.AccountingByteBuf.writeBytes(AccountingByteBuf.java:664)
> at org.apache.drill.exec.store.parquet.NullableFixedByteAlignedReader.readField(NullableFixedByteAlignedReader.java:48)
> at org.apache.drill.exec.store.parquet.NullableColumnReader.readAllFixedFields(NullableColumnReader.java:106)
> at org.apache.drill.exec.store.parquet.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:327)
> at org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:369)
> at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:94)
> at org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
> at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:85)
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:83)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
--
This message was sent by Atlassian JIRA
(v6.2#6252)