You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Steven Phillips <sp...@maprtech.com> on 2015/05/11 14:11:03 UTC
Review Request 34037: Patch for DRILL-2936
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34037/
-----------------------------------------------------------
Review request for drill.
Bugs: DRILL-2936
https://issues.apache.org/jira/browse/DRILL-2936
Repository: drill-git
Description
-------
DRILL-2936: Use SpoolingRawBatchBuffer for HashToMergeExchange In order to avoid deadlocks
Refactored common code in UnlimitedRawBatchBuffer and SpoolingRawBatchBuffer
into BaseRawBatchBuffer
Removed reflection-based construction of RawBatchBuffer. Now use choose implementation
based on plan
Updated SpoolingRawBatchBuffer to use a separate thread for spooling
Diffs
-----
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractReceiver.java f01d025e8d7a9e3c4fe907a1c134c951b258dc9b
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/Receiver.java 04d6d7eea28f6ada994384a6edbace2e1bf3d863
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/BroadcastExchange.java a37f638bc0fe71f61eee28e05777f636d21a0822
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java f45ace92357dd4ad3f787a17e32f6ad755a0794c
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToRandomExchange.java 52d79c22603343f8dfe66bc08817e7486f4198c5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java 9416814375a39419c25a3465af9dac59c179f0f3
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/OrderedPartitionExchange.java c8dbc225d148deee791c96ff8f602d59f8104daf
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java c812325807bdec70f1af7c1373ca1ff40e7ac29f
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnionExchange.java b7b7835282f529e68256ea014b4da4ea9d269b87
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedDeMuxExchange.java 0bc6678eddc9faa4d6b09ec378adb7712cde3960
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedMuxExchange.java 3028ee3ed2bcb778fe41e393af7ee1f0198d1cec
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedReceiver.java e741dd422543a63ba9d7dcea54f822f03870be48
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java b88cc28d2f89dcf0004809ecedcef06e5ab187d0
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java ed163146d95ac0d5a12bf246765da373668fec84
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 07a3505f105ad01c392d834a8df2e2075c676d53
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 475066661c7629d7b50741cf9413977f5d8cfcc6
exec/java-exec/src/main/resources/drill-module.conf d98b97a7cfd9c0667d16832b35a44368562346f6
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java dcea9bbecf1b7f068de61bfe897586412cfabcc0
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java b8336e9779d45251081128ff1450adc4a2f38576
Diff: https://reviews.apache.org/r/34037/diff/
Testing
-------
Thanks,
Steven Phillips
Re: Review Request 34037: Patch for DRILL-2936
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34037/#review83671
-----------------------------------------------------------
Ship it!
Ship It!
- Jacques Nadeau
On May 13, 2015, 12:34 a.m., Steven Phillips wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34037/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 12:34 a.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-2936
> https://issues.apache.org/jira/browse/DRILL-2936
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2936: Use SpoolingRawBatchBuffer for HashToMergeExchange In order to avoid deadlocks
>
> Refactored common code in UnlimitedRawBatchBuffer and SpoolingRawBatchBuffer
> into BaseRawBatchBuffer
>
> Removed reflection-based construction of RawBatchBuffer. Now use choose implementation
> based on plan
>
> Updated SpoolingRawBatchBuffer to use a separate thread for spooling
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractReceiver.java f01d025e8d7a9e3c4fe907a1c134c951b258dc9b
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/Receiver.java 04d6d7eea28f6ada994384a6edbace2e1bf3d863
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/BroadcastExchange.java a37f638bc0fe71f61eee28e05777f636d21a0822
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java f45ace92357dd4ad3f787a17e32f6ad755a0794c
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToRandomExchange.java 52d79c22603343f8dfe66bc08817e7486f4198c5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java 9416814375a39419c25a3465af9dac59c179f0f3
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/OrderedPartitionExchange.java c8dbc225d148deee791c96ff8f602d59f8104daf
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java c812325807bdec70f1af7c1373ca1ff40e7ac29f
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnionExchange.java b7b7835282f529e68256ea014b4da4ea9d269b87
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedDeMuxExchange.java 0bc6678eddc9faa4d6b09ec378adb7712cde3960
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedMuxExchange.java 3028ee3ed2bcb778fe41e393af7ee1f0198d1cec
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedReceiver.java e741dd422543a63ba9d7dcea54f822f03870be48
> exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java edd79ac818c490a344c929c7e40a7502ef93300e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java b88cc28d2f89dcf0004809ecedcef06e5ab187d0
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java ed163146d95ac0d5a12bf246765da373668fec84
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java 8646a72e4789a038b331509e8cdef6af20d77851
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 07a3505f105ad01c392d834a8df2e2075c676d53
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 475066661c7629d7b50741cf9413977f5d8cfcc6
> exec/java-exec/src/main/resources/drill-module.conf d98b97a7cfd9c0667d16832b35a44368562346f6
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java dcea9bbecf1b7f068de61bfe897586412cfabcc0
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java b8336e9779d45251081128ff1450adc4a2f38576
>
> Diff: https://reviews.apache.org/r/34037/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Steven Phillips
>
>
Re: Review Request 34037: Patch for DRILL-2936
Posted by Steven Phillips <sp...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34037/
-----------------------------------------------------------
(Updated May 13, 2015, 12:34 a.m.)
Review request for drill.
Bugs: DRILL-2936
https://issues.apache.org/jira/browse/DRILL-2936
Repository: drill-git
Description
-------
DRILL-2936: Use SpoolingRawBatchBuffer for HashToMergeExchange In order to avoid deadlocks
Refactored common code in UnlimitedRawBatchBuffer and SpoolingRawBatchBuffer
into BaseRawBatchBuffer
Removed reflection-based construction of RawBatchBuffer. Now use choose implementation
based on plan
Updated SpoolingRawBatchBuffer to use a separate thread for spooling
Diffs (updated)
-----
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractReceiver.java f01d025e8d7a9e3c4fe907a1c134c951b258dc9b
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/Receiver.java 04d6d7eea28f6ada994384a6edbace2e1bf3d863
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/BroadcastExchange.java a37f638bc0fe71f61eee28e05777f636d21a0822
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java f45ace92357dd4ad3f787a17e32f6ad755a0794c
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToRandomExchange.java 52d79c22603343f8dfe66bc08817e7486f4198c5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java 9416814375a39419c25a3465af9dac59c179f0f3
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/OrderedPartitionExchange.java c8dbc225d148deee791c96ff8f602d59f8104daf
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java c812325807bdec70f1af7c1373ca1ff40e7ac29f
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnionExchange.java b7b7835282f529e68256ea014b4da4ea9d269b87
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedDeMuxExchange.java 0bc6678eddc9faa4d6b09ec378adb7712cde3960
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedMuxExchange.java 3028ee3ed2bcb778fe41e393af7ee1f0198d1cec
exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnorderedReceiver.java e741dd422543a63ba9d7dcea54f822f03870be48
exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java edd79ac818c490a344c929c7e40a7502ef93300e
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java b88cc28d2f89dcf0004809ecedcef06e5ab187d0
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java ed163146d95ac0d5a12bf246765da373668fec84
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java 8646a72e4789a038b331509e8cdef6af20d77851
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 07a3505f105ad01c392d834a8df2e2075c676d53
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 475066661c7629d7b50741cf9413977f5d8cfcc6
exec/java-exec/src/main/resources/drill-module.conf d98b97a7cfd9c0667d16832b35a44368562346f6
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java dcea9bbecf1b7f068de61bfe897586412cfabcc0
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestUnlimitedBatchBuffer.java b8336e9779d45251081128ff1450adc4a2f38576
Diff: https://reviews.apache.org/r/34037/diff/
Testing
-------
Thanks,
Steven Phillips