You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Aman Sinha <as...@maprtech.com> on 2015/03/19 18:51:57 UTC
Review Request 32244: DRILL-2353: Handle more complex partition
filters via interpreter based partition pruning
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32244/
-----------------------------------------------------------
Review request for drill and Jacques Nadeau.
Repository: drill-git
Description
-------
- Integrate Jacques's interpreter based partition pruning with Jason's interpreter refactoring that removed interpreter module and added that functionality in the exec module.
- Rewrite FindPartitionFilter to handle ORs and ANDs correctly: This is done by maintaining a separate stack of BooleanOpState which keeps track of the boolean operators and its children.
Diffs
-----
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/planner/sql/HivePartitionDescriptor.java e6ca21e
exec/interpreter/src/test/java/org/apache/drill/exec/expr/TestPrune.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java 4f8e126
exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java 4c1f8e8
exec/java-exec/src/main/java/org/apache/drill/exec/planner/PartitionDescriptor.java 02a6a8f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 496bc9a
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/FindPartitionConditions.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/expr/TestPrune.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/planner/logical/FilterSplitTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/32244/diff/
Testing
-------
Unit tests, functional and tpch sf100 tests. The partition pruning functional tests have several plan changes: Two main classes of differences are: we won't drop the filter node anymore even when all directory filters are pushed, and the order of the tables in the scan is different in some cases. So baselines would need to be updated.
Thanks,
Aman Sinha
Re: Review Request 32244: DRILL-2353: Handle more complex partition
filters via interpreter based partition pruning
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32244/#review77165
-----------------------------------------------------------
Ship it!
Ship It!
- Jacques Nadeau
On March 19, 2015, 5:51 p.m., Aman Sinha wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32244/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 5:51 p.m.)
>
>
> Review request for drill and Jacques Nadeau.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> - Integrate Jacques's interpreter based partition pruning with Jason's interpreter refactoring that removed interpreter module and added that functionality in the exec module.
>
> - Rewrite FindPartitionFilter to handle ORs and ANDs correctly: This is done by maintaining a separate stack of BooleanOpState which keeps track of the boolean operators and its children.
>
>
> Diffs
> -----
>
> contrib/storage-hive/core/src/main/java/org/apache/drill/exec/planner/sql/HivePartitionDescriptor.java e6ca21e
> exec/interpreter/src/test/java/org/apache/drill/exec/expr/TestPrune.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java 4f8e126
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java 4c1f8e8
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/PartitionDescriptor.java 02a6a8f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 496bc9a
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/FindPartitionConditions.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/expr/TestPrune.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/planner/logical/FilterSplitTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/32244/diff/
>
>
> Testing
> -------
>
> Unit tests, functional and tpch sf100 tests. The partition pruning functional tests have several plan changes: Two main classes of differences are: we won't drop the filter node anymore even when all directory filters are pushed, and the order of the tables in the scan is different in some cases. So baselines would need to be updated.
>
>
> Thanks,
>
> Aman Sinha
>
>