You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Aman Sinha (JIRA)" <ji...@apache.org> on 2014/07/07 20:48:35 UTC
[jira] [Commented] (DRILL-1111) IllegalArgumentException for TPCH
10 on scale factor SF100
[ https://issues.apache.org/jira/browse/DRILL-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054010#comment-14054010 ]
Aman Sinha commented on DRILL-1111:
-----------------------------------
The problem was that for long varchar columns in the group-by we were running out of space when either storing the keys in the hash table or when outputting the keys and values to the output batch. I have uploaded the patch with the following changes:
1. Create new batch holder for hash table if an insertion fails due to space constraints in existing batch holder.
2. Use allocateNew() for hash aggr and hash table allocations.
3. Create new output batch if output values fails the first time due to space constraints. 4. Use splitAndTransfer to transfer the keys from hash table's container to the output batch.
> IllegalArgumentException for TPCH 10 on scale factor SF100
> -----------------------------------------------------------
>
> Key: DRILL-1111
> URL: https://issues.apache.org/jira/browse/DRILL-1111
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Attachments: 0001-DRILL-1111.patch
>
>
> TPCH 10 on SF100 scale factor gives an IllegalArgumentException when attempting to output keys from the hash table:
> java.lang.IllegalArgumentException: length: -3276780 (expected: >= 0)
> at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1127) ~[netty-buffer-4.0.7.Final.jar:na]
> at io.netty.buffer.PooledUnsafeDirectByteBufL.getBytes(PooledUnsafeDirectByteBufL.java:120) ~[drill-buffers-1.0.0-m2-incubating-SNAPSHOT.jar:na]
> at org.apache.drill.exec.memory.AccountingByteBuf.getBytes(AccountingByteBuf.java:322) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at org.apache.drill.exec.vector.VarBinaryVector$Mutator.setSafe(VarBinaryVector.java:472) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.setSafe(NullableVarBinaryVector.java:429) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at org.apache.drill.exec.test.generated.HashTableGen44$BatchHolder.outputRecordKeys(HashTableTemplate.java:1368) ~[na:na]
> at org.apache.drill.exec.test.generated.HashTableGen44$BatchHolder.outputKeys(HashTableTemplate.java:265) ~[na:na]
> at org.apache.drill.exec.test.generated.HashTableGen44$BatchHolder.access$1400(HashTableTemplate.java:97) ~[na:na]
> at org.apache.drill.exec.test.generated.HashTableGen44.outputKeys(HashTableTemplate.java:627) ~[na:na]
> at org.apache.drill.exec.test.generated.HashAggregatorGen43.outputCurrentBatch(HashAggTemplate.java:439) ~[na:na]
> at org.apache.drill.exec.test.generated.HashAggregatorGen43.doWork(HashAggTemplate.java:293) ~[na:na]
> at org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:140) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHO
--
This message was sent by Atlassian JIRA
(v6.2#6252)