[jira] [Commented] (TAJO-666) java.nio.BufferOverflowException occurs when the query includes an order by clause on a TEXT column

["Mai Hai Thanh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TAJO-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068301#comment-14068301 ] 

Mai Hai Thanh commented on TAJO-666:
------------------------------------

Hi,

I created a patch to solve to this bug. Instead of allocating a fixed ByteBuffer of 4096 bytes, we estimate the size of row data and then use this estimated value for allocating the ByteBuffer. This solution not only eliminates the BufferOverflowException but also reduces the memory usage in many cases (4096 bytes is often too big for short rows, especially the ones that do not contain any TEXT value).

> java.nio.BufferOverflowException occurs when the query includes an order by clause on a TEXT column
> ---------------------------------------------------------------------------------------------------
>
>                 Key: TAJO-666
>                 URL: https://issues.apache.org/jira/browse/TAJO-666
>             Project: Tajo
>          Issue Type: Bug
>            Reporter: Jihoon Son
>
> See the title.
> The query and the error log of the query master are as follows.
> The 'method' column type is TEXT.
> {noformat}
> tajo> select method, sum(totaltime) from logs_20130404 group by method order by method;
> 2014-03-06 11:26:03,432 ERROR querymaster.QueryUnitAttempt (QueryUnitAttempt.java:transition(409)) - FROM xxx.xxx.xxx.xxx >> java.nio.BufferOverflowException
>         at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)
>         at java.nio.ByteBuffer.put(ByteBuffer.java:832)
>         at org.apache.tajo.storage.RowStoreUtil$RowStoreEncoder.toBytes(RowStoreUtil.java:181)
>         at org.apache.tajo.storage.index.bst.BSTIndex$BSTIndexWriter.writeHeader(BSTIndex.java:168)
>         at org.apache.tajo.storage.index.bst.BSTIndex$BSTIndexWriter.close(BSTIndex.java:187)
>         at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:237)
>         at org.apache.tajo.engine.planner.physical.RangeShuffleFileWriteExec.close(RangeShuffleFileWriteExec.java:123)
>         at org.apache.tajo.worker.Task.run(Task.java:385)
>         at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:395)
>         at java.lang.Thread.run(Thread.java:744)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)