You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tajo.apache.org by "Jihoon Son (JIRA)" <ji...@apache.org> on 2013/09/06 13:54:51 UTC
[jira] [Comment Edited] (TAJO-156) BufferUnderflowException occurs
during processing the intermediate file
[ https://issues.apache.org/jira/browse/TAJO-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760156#comment-13760156 ]
Jihoon Son edited comment on TAJO-156 at 9/6/13 11:53 AM:
----------------------------------------------------------
BufferUnderflowException still occurs during processing the sort query.
Here are a log and the query.
{code}
tajo> \d table
table name: table
table path: hdfs:///path/to/table
store type: CSV
number of rows: 0
volume (bytes): 423.7 MB
schema:
col0 FLOAT8
col1 INT4
col2 TEXT
col3 TEXT
col4 INT4
col5 INT4
col6 TEXT
col7 TEXT
col8 TEXT
col9 TEXT
col10 TEXT
col11 TEXT
tajo> SELECT col10 , sum(col5) as sum_int FROM table GROUP BY col10 ORDER BY sum_int desc;
{code}
I changed the code to see the status of ByteBuffer.
This exception is occurred during the RawFile reads text type data.
As seen in the log, the length of the value is too large and this caused the BufferUnderflowException.
{code}
13/09/06 20:11:17 ERROR querymaster.QueryUnitAttempt: FROM 127.0.0.1 >> java.io.IOException: length: 458888 pos: 68300 remain: 141 limit: 68441
at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:206)
at org.apache.tajo.engine.planner.physical.ExternalSortExec.next(ExternalSortExec.java:181)
at org.apache.tajo.engine.planner.physical.IndexedStoreExec.next(IndexedStoreExec.java:91)
at org.apache.tajo.worker.Task.run(Task.java:378)
at org.apache.tajo.worker.TaskRunner$2.run(TaskRunner.java:359)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.nio.BufferUnderflowException
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:239)
at java.nio.ByteBuffer.get(ByteBuffer.java:692)
at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:203)
... 5 more
{code}
was (Author: jihoonson):
BufferUnderflowException still occurs during processing the sort query.
Here are a log and the query.
{code}
tajo> \d table
table name: table
table path: hdfs:///path/to/table
store type: CSV
number of rows: 0
volume (bytes): 423.7 MB
schema:
col0 FLOAT8
col1 INT4
col2 TEXT
col3 TEXT
col4 INT4
col5 INT4
col6 TEXT
col7 TEXT
col8 TEXT
col9 TEXT
col10 TEXT
col11 TEXT
tajo> SELECT col10 , sum(col5) as sum_int FROM table GROUP BY col10 ORDER BY sum_int desc;
{code}
As seen in the log, I changed the code to see the status of ByteBuffer.
{code}
13/09/06 20:11:17 ERROR querymaster.QueryUnitAttempt: FROM 127.0.0.1 >> java.io.IOException: length: 458888 pos: 68300 remain: 141 limit: 68441
at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:206)
at org.apache.tajo.engine.planner.physical.ExternalSortExec.next(ExternalSortExec.java:181)
at org.apache.tajo.engine.planner.physical.IndexedStoreExec.next(IndexedStoreExec.java:91)
at org.apache.tajo.worker.Task.run(Task.java:378)
at org.apache.tajo.worker.TaskRunner$2.run(TaskRunner.java:359)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.nio.BufferUnderflowException
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:239)
at java.nio.ByteBuffer.get(ByteBuffer.java:692)
at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:203)
... 5 more
{code}
> BufferUnderflowException occurs during processing the intermediate file
> -----------------------------------------------------------------------
>
> Key: TAJO-156
> URL: https://issues.apache.org/jira/browse/TAJO-156
> Project: Tajo
> Issue Type: Bug
> Components: storage
> Reporter: Jinho Kim
> Fix For: 0.2-incubating
>
> Attachments: TAJO-156_2.patch, TAJO-156.patch
>
>
> {noformat}
> 2013-09-04 18:30:53,701 ERROR worker.Task (Task.java:run(383)) - java.nio.BufferUnderflowException
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:260)
> at java.nio.ByteBuffer.get(ByteBuffer.java:675)
> at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:196)
> at org.apache.tajo.engine.planner.physical.ExternalSortExec.next(ExternalSortExec.java:175)
> at org.apache.tajo.engine.planner.physical.SortAggregateExec.next(SortAggregateExec.java:46)
> at org.apache.tajo.engine.planner.physical.PartitionedStoreExec.next(PartitionedStoreExec.java:124)
> at org.apache.tajo.worker.Task.run(Task.java:375)
> at org.apache.tajo.worker.TaskRunner$2.run(TaskRunner.java:377)
> at java.lang.Thread.run(Thread.java:680)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira