You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jason Altekruse <al...@gmail.com> on 2014/11/07 23:53:55 UTC

Re: Review Request 27257: Drill-1333 - Flatten operator

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27257/
-----------------------------------------------------------

(Updated Nov. 7, 2014, 10:53 p.m.)


Review request for drill and Mehant Baid.


Changes
-------

This is a patch that includes the enabling of the second re-write rule a well as a few fixes, while the majority of this patch has been committed this is being used to review the re-write rule that was committed but disabled.


Repository: drill-git


Description
-------

Flatten operation to allow for more fleible exploration and querying of repeated data. Can be used together with recently added kvgen function to run queries against maps as well.


Diffs (updated)
-----

  exec/java-exec/src/main/codegen/templates/BaseWriter.java 224148c 
  exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 2132212 
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java d1e10ae 
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/DummyFlatten.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 4b290a5 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractPhysicalVisitor.java 031ab10 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/PhysicalVisitor.java f9a9c21 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/FlattenPOP.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenBatchCreator.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenTemplate.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/Flattener.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillFlattenPrel.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java a036a46 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ProjectPrel.java 1f621da 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/BasePrelVisitor.java 81f247d 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/PrelVisitor.java f519220 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/RewriteProjectToFlatten.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/RexVisitorComplexExprSplitter.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/SplitUpComplexExpressions.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java 0bb59bf 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedFixedWidthVector.java a28caf1 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVariableWidthVector.java 2e7689c 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVector.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java 9870dd5 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java 2612924 
  exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/AbstractBaseWriter.java 7aa9846 
  exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java 3a195e8 
  exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java ff7ba44 
  exec/java-exec/src/test/resources/flatten/corrected_physical.json PRE-CREATION 
  exec/java-exec/src/test/resources/flatten/test_flatten_physical.json PRE-CREATION 
  exec/java-exec/src/test/resources/jsoninput/input2_modified.json PRE-CREATION 
  exec/java-exec/src/test/resources/jsoninput/repeated_list_bug.json PRE-CREATION 
  exec/java-exec/src/test/resources/store/json/nested_json.json PRE-CREATION 
  exec/java-exec/src/test/resources/store/json/test_flatten_mapify.json PRE-CREATION 

Diff: https://reviews.apache.org/r/27257/diff/


Testing
-------

Unit tests pass, waiting on some debugging for the regression and TPCH test cluster to finish to run those tests, but wanted to allow the review to start. 


Thanks,

Jason Altekruse