You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jason Altekruse <al...@gmail.com> on 2015/04/09 06:36:55 UTC
Review Request 32980: Drill-2162: flatten violating batch size limit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32980/
-----------------------------------------------------------
Review request for drill and Mehant Baid.
Bugs: DRILL-2162
https://issues.apache.org/jira/browse/DRILL-2162
Repository: drill-git
Description
-------
Flatten did not have a limitiation on the size of the outgoing record batch. Previously this was not noticed even flattening files with large lists because the reading of files used to produce fewer records before automatic vector reallocation was introduced. While the introduction of the reallocation feature exposed this issue, all operators are expected to manage this themselves and thus flatten is the correct place the fix the issue.
Diffs
-----
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenTemplate.java c5d3d93
exec/java-exec/src/test/java/org/apache/drill/exec/fn/interp/TestConstantFolding.java b17935a
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/flatten/TestFlatten.java 6f5a303
exec/java-exec/src/test/resources/flatten/complex_transaction_example_data.json PRE-CREATION
Diff: https://reviews.apache.org/r/32980/diff/
Testing
-------
Thanks,
Jason Altekruse
Re: Review Request 32980: Drill-2162: flatten violating batch size
limit
Posted by Mehant Baid <ba...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32980/#review79607
-----------------------------------------------------------
Ship it!
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenTemplate.java
<https://reviews.apache.org/r/32980/#comment129106>
With the reallocation changes do you think doEval() will ever return false?
- Mehant Baid
On April 9, 2015, 4:36 a.m., Jason Altekruse wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32980/
> -----------------------------------------------------------
>
> (Updated April 9, 2015, 4:36 a.m.)
>
>
> Review request for drill and Mehant Baid.
>
>
> Bugs: DRILL-2162
> https://issues.apache.org/jira/browse/DRILL-2162
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Flatten did not have a limitiation on the size of the outgoing record batch. Previously this was not noticed even flattening files with large lists because the reading of files used to produce fewer records before automatic vector reallocation was introduced. While the introduction of the reallocation feature exposed this issue, all operators are expected to manage this themselves and thus flatten is the correct place the fix the issue.
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenTemplate.java c5d3d93
> exec/java-exec/src/test/java/org/apache/drill/exec/fn/interp/TestConstantFolding.java b17935a
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/flatten/TestFlatten.java 6f5a303
> exec/java-exec/src/test/resources/flatten/complex_transaction_example_data.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/32980/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jason Altekruse
>
>