You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Lisen Mu <im...@gmail.com> on 2013/08/01 04:14:25 UTC
Review Request 13183: Reentrant physical operator/multiple iteration over record batch

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13183/
-----------------------------------------------------------

Review request for drill.


Repository: drill-git


Description
-------

Reentrant physical operator implementation.

IterationBuffer acts as a proxy which can be applied to any other RecordBatch. IterationBuffer keeps iteration result sequence of its incoming RecordBatch, and supports multiple iteration over this buffer.

so the batch chain:

  ScreenBatch -> FilterBatch -> ScanBatch

becomes:

  ScreenBatch -> BufferedRecordBatch --(its internal buffer)--> FilterBatch -> BufferedRecordBatch --(internal buffer)--> ScanBatch

The diff includes Union commits to run test.


Diffs
-----

  sandbox/prototype/exec/java-exec/src/main/codegen/ValueVectors/templates/FixedValueVectors.java 61515b04f33617320562769cd71163e9e1d5949b 
  sandbox/prototype/exec/java-exec/src/main/codegen/ValueVectors/templates/NullableValueVectors.java dfec62e1a5fa2392c50d016d5e618cc930f17b18 
  sandbox/prototype/exec/java-exec/src/main/codegen/ValueVectors/templates/RepeatedValueVectors.java 30bc0860965275f06c989c5e8d749578afe8058b 
  sandbox/prototype/exec/java-exec/src/main/codegen/ValueVectors/templates/VariableLengthVectors.java 7daadee348db7ee38707c2ef67f44721cb8ce521 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractBase.java e91257e9d1b25eccb2562ff12992f77ebbc3e981 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractMultiple.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractPhysicalVisitor.java f7823257afe52ed15871e38a617e61f7fb516fb8 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/PhysicalVisitor.java f36633fa889e20f2afb3519b7cda86900c3fc022 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/reentrant/ReentrantDelegate.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/reentrant/ReentrantOperator.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/reentrant/ReentrantPhysicalOperator.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/Filter.java 4af424350d2241d501e7a79f81f4ced44962a5b6 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/Union.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ImplCreator.java 77409554e70a6d797f69b59ac52fca0b7f039e5c 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java 516b5af33773bd7b48442a5d4e3c266e5ce536bf 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/BufferedBatchCreator.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterBatchCreator.java df2518b99a43e7248f6d4daa4ed741fcb08fb6e2 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionBatchCreator.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionRecordBatch.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/TransferPair.java a90e2d8a93549f9f9b6a0ec29d0d3c57025897ad 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/buffered/BufferedRecordBatch.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/buffered/BufferedStackFrame.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/buffered/IterationBuffer.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 0f5b7d006bdae4d2e826b61350de018c09a61a8e 
  sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/vector/ValueVector.java 9328a805ab186ba7eddac6ce9d3fca5c2170be8d 
  sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFragmentRun.java 7b002ea96865d5e024005cbd733b9627b9c8d090 
  sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java 96a6139d57a92d212e54aca4ed1a5299fa177676 
  sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/test/resources/filter/test1.json c9b367f5ea4adda0c0d34437f0bdf6b4215d00e9 
  sandbox/prototype/exec/java-exec/src/test/resources/filter/test2.json PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/test/resources/project/test1.json 70a51151839eace0a8c944f9a47ebd8465a3d2c7 
  sandbox/prototype/exec/java-exec/src/test/resources/union/test1.json PRE-CREATION 
  sandbox/prototype/exec/java-exec/src/test/resources/union/test2_dag.json PRE-CREATION 

Diff: https://reviews.apache.org/r/13183/diff/


Testing
-------


TestSimpleUnion, with /union/test2_dag.json


Thanks,

Lisen Mu