You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jacques Nadeau <ja...@gmail.com> on 2015/03/06 19:46:44 UTC
Review Request 31804: DRILL-2187: Broadcast join planning and shared
data distribution
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31804/
-----------------------------------------------------------
Review request for drill and Aman Sinha.
Repository: drill-git
Description
-------
Update the RPC layer so that we can do multi-target broadcast sends.
Update planning layer to correctly propagate and explore traits using subsets rather than inner children.
Diffs
-----
exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 8e9d395
exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java 3de0a75
exec/java-exec/src/main/java/org/apache/drill/exec/memory/Accountor.java 2b48ef0
exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 83d9d1e
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 67e1fdb
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java 6db9f4a
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java 22fa047
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java a23bd7a
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java 71ffd41
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 3b7adca
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java 0c76de4
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTraitDef.java 6a1dbb7
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrule.java 433405a
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/JoinPruleBase.java afcbf71
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrel.java b7e86e3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrule.java faffa63
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java bbfbbcb
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ProjectPrule.java 72034ed
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java 89b133a
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SubsetTransformer.java d4cd21f
exec/java-exec/src/main/java/org/apache/drill/exec/proto/helper/QueryIdHelper.java bb12a22
exec/java-exec/src/main/java/org/apache/drill/exec/record/FragmentWritableBatch.java 3d06806
exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java e1725e6
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcBus.java 96c9911
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/AckSender.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataClient.java a9eb66f
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataConnectionCreator.java 197996d
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataConnectionManager.java b2ea855
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataProtobufLengthDecoder.java 193b050
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataResponseHandler.java d4a73c3
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataResponseHandlerImpl.java 1fcb3e9
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java b54841d
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java e88455b
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 3d3e96f
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ResponseSenderQueue.java 141c434
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 6ee93ab
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 895918c
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/broadcastsender/TestBroadcast.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 7b771f5
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java a710d21
exec/java-exec/src/test/resources/broadcast/customer/cust.json PRE-CREATION
exec/java-exec/src/test/resources/broadcast/sales/f1.json PRE-CREATION
exec/java-exec/src/test/resources/broadcast/sales/f2.json PRE-CREATION
protocol/src/main/java/org/apache/drill/exec/proto/BitData.java 37e8a18
protocol/src/main/java/org/apache/drill/exec/proto/SchemaBitData.java f35f489
protocol/src/main/java/org/apache/drill/exec/proto/beans/BitClientHandshake.java 55810fe
protocol/src/main/java/org/apache/drill/exec/proto/beans/FragmentRecordBatch.java eac819c
protocol/src/main/protobuf/BitData.proto de8e9e7
Diff: https://reviews.apache.org/r/31804/diff/
Testing
-------
Thanks,
Jacques Nadeau
Re: Review Request 31804: DRILL-2187: Broadcast join planning and
shared data distribution
Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31804/#review75518
-----------------------------------------------------------
exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java
<https://reviews.apache.org/r/31804/#comment122650>
assert decrement > 0?
exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java
<https://reviews.apache.org/r/31804/#comment122651>
assert increment > 0?
- Timothy Chen
On March 6, 2015, 6:46 p.m., Jacques Nadeau wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31804/
> -----------------------------------------------------------
>
> (Updated March 6, 2015, 6:46 p.m.)
>
>
> Review request for drill and Aman Sinha.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Update the RPC layer so that we can do multi-target broadcast sends.
> Update planning layer to correctly propagate and explore traits using subsets rather than inner children.
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 8e9d395
> exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java 3de0a75
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/Accountor.java 2b48ef0
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 83d9d1e
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 67e1fdb
> exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java 6db9f4a
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java 22fa047
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java a23bd7a
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java 71ffd41
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 3b7adca
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java 0c76de4
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTraitDef.java 6a1dbb7
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrule.java 433405a
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/JoinPruleBase.java afcbf71
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrel.java b7e86e3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrule.java faffa63
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java bbfbbcb
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ProjectPrule.java 72034ed
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java 89b133a
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SubsetTransformer.java d4cd21f
> exec/java-exec/src/main/java/org/apache/drill/exec/proto/helper/QueryIdHelper.java bb12a22
> exec/java-exec/src/main/java/org/apache/drill/exec/record/FragmentWritableBatch.java 3d06806
> exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java e1725e6
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcBus.java 96c9911
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/AckSender.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataClient.java a9eb66f
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataConnectionCreator.java 197996d
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataConnectionManager.java b2ea855
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataProtobufLengthDecoder.java 193b050
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataResponseHandler.java d4a73c3
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataResponseHandlerImpl.java 1fcb3e9
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java b54841d
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java e88455b
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 3d3e96f
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ResponseSenderQueue.java 141c434
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 6ee93ab
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 895918c
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/broadcastsender/TestBroadcast.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/server/TestBitRpc.java 7b771f5
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java a710d21
> exec/java-exec/src/test/resources/broadcast/customer/cust.json PRE-CREATION
> exec/java-exec/src/test/resources/broadcast/sales/f1.json PRE-CREATION
> exec/java-exec/src/test/resources/broadcast/sales/f2.json PRE-CREATION
> protocol/src/main/java/org/apache/drill/exec/proto/BitData.java 37e8a18
> protocol/src/main/java/org/apache/drill/exec/proto/SchemaBitData.java f35f489
> protocol/src/main/java/org/apache/drill/exec/proto/beans/BitClientHandshake.java 55810fe
> protocol/src/main/java/org/apache/drill/exec/proto/beans/FragmentRecordBatch.java eac819c
> protocol/src/main/protobuf/BitData.proto de8e9e7
>
> Diff: https://reviews.apache.org/r/31804/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jacques Nadeau
>
>