You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Steven Phillips <sp...@maprtech.com> on 2013/11/15 05:00:05 UTC
Review Request 15564: Patch for DRILL-274
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15564/
-----------------------------------------------------------
Review request for drill.
Bugs: DRILL-274
https://issues.apache.org/jira/browse/DRILL-274
Repository: drill-git
Description
-------
DRILL-274: Spooling batch buffer
Diffs
-----
distribution/src/resources/drill-override.conf 7694cedf62ae5dde1221d3bef0ac6cdaedaf78d4
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java a7f6d2ef03eb2fef20fed5f327a12ea3c82989de
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java ac05319725e0bf797a67a133f44a9a6988ab12c2
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java c47fa89ab65b18ca21fd4b2f2dc9d478b5fe4809
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 7e7ac633aacd70469cbf83f4b52641853d690f1f
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java cbb3e5495fdf0a3efc093646c461b9216e2bc486
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java 7ce69e2bf7248aa78444d89524c3d4d06e275835
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java e81774ad74f2eeeeb78e68216c5ab0a3b60f6219
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java ceebe559f5f9d605aab13a1a204443249c012292
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java ef7638ede343b055a6686ffaa7f7d6d72a97c118
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java 884887d7de1542974766185b3c9675af7dcfccbd
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java fd813b41224b309669ffdbe6c1bd1662458fe3b3
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java 634182a57a1533d8410b086c4e4b7ca18a23041a
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java ea14b08d6267e2fb1b1e950d089d94320edac3f1
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java 4c04abcd38c5d77f0b8da7f263ed220c0a568ddf
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java 92c745fa08cae70b5ed991b87ce751613f0cbfdc
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java PRE-CREATION
exec/java-exec/src/test/resources/drill-module.conf 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
exec/java-exec/src/test/resources/drill-spool-test-module.conf PRE-CREATION
exec/java-exec/src/test/resources/work/batch/multiple_exchange.json PRE-CREATION
Diff: https://reviews.apache.org/r/15564/diff/
Testing
-------
Thanks,
Steven Phillips
Re: Review Request 15564: Patch for DRILL-274
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15564/#review29251
-----------------------------------------------------------
Ship it!
Ship It!
- Jacques Nadeau
On Nov. 19, 2013, 1:57 a.m., Steven Phillips wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15564/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2013, 1:57 a.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-274
> https://issues.apache.org/jira/browse/DRILL-274
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Lazily initialize input/ouptut streams
>
>
> DRILL-274: resolve merge conflict
>
>
> DRILL-274: Address code review comments.
>
>
> DRILL-274: Spooling batch buffer
>
>
> Diffs
> -----
>
> distribution/src/resources/drill-override.conf 18a2a6a80784951ab8c78b58698f65f0575aa783
> exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
> exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 674dafca0c8b7840f4af0edf5b34741b3a06027d
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WireRecordBatch.java 0b0214ac610e949a0d23ff16f3f5f0756a330f77
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java 6d8a28424e3248c21876a1e1c5600ea5855c681b
> exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatchProvider.java 3390af9c354c5b5d4fc56727bc7723c9bb3c6e07
> exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BatchCollector.java 236b2398f36d6e22ae83f6be0095ef8d64186be7
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java 82ed1ca1f17f9be93f7cb1863000b3e564417222
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
> exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
> exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
> exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java a78ffc310506c067b0139e24cb5683adf8549324
> exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java 3ec94929d708f4d5f02d657162369f3dbf1920c6
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java 54bf0fd29651bb179af872c0dad436f43e260fb8
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 05d57be80a4120949fc40799d023eeab04447f91
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java 1776d8df4310b0f43edc1df4546ce6839548c127
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java b18ef712879362fad3aa9403b103bec87fbe2522
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java 14d68f386c1f673135b35185cb03bcf7d0cc1a74
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java 09b7ebed255e28b387763cf9e70c0a621e936cb9
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java 89d909dfe735321d15d8c39dcdc97a02ae97c818
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java ca38d9c1963f366b558750377130c72045709031
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java 426aa3a70397ca409557f13260c9a26b3d0a2bbb
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java 2da96d770dd4212d13390fced0bdab21720326dd
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java dd7d00636bbe2fec267d586165814a14552e17e6
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java f4e61806bc1e6b75b6114c4709021bfb8b7e29b9
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java f74c66f6d75c65075ba2e209bc138462c9a26b36
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java PRE-CREATION
> exec/java-exec/src/test/resources/drill-module.conf 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
> exec/java-exec/src/test/resources/drill-spool-test-module.conf PRE-CREATION
> exec/java-exec/src/test/resources/work/batch/multiple_exchange.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15564/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Steven Phillips
>
>
Re: Review Request 15564: Patch for DRILL-274
Posted by Steven Phillips <sp...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15564/
-----------------------------------------------------------
(Updated Nov. 19, 2013, 1:57 a.m.)
Review request for drill.
Bugs: DRILL-274
https://issues.apache.org/jira/browse/DRILL-274
Repository: drill-git
Description (updated)
-------
Lazily initialize input/ouptut streams
DRILL-274: resolve merge conflict
DRILL-274: Address code review comments.
DRILL-274: Spooling batch buffer
Diffs (updated)
-----
distribution/src/resources/drill-override.conf 18a2a6a80784951ab8c78b58698f65f0575aa783
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 674dafca0c8b7840f4af0edf5b34741b3a06027d
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WireRecordBatch.java 0b0214ac610e949a0d23ff16f3f5f0756a330f77
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java 6d8a28424e3248c21876a1e1c5600ea5855c681b
exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatchProvider.java 3390af9c354c5b5d4fc56727bc7723c9bb3c6e07
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BatchCollector.java 236b2398f36d6e22ae83f6be0095ef8d64186be7
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java 82ed1ca1f17f9be93f7cb1863000b3e564417222
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java a78ffc310506c067b0139e24cb5683adf8549324
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java 3ec94929d708f4d5f02d657162369f3dbf1920c6
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java 54bf0fd29651bb179af872c0dad436f43e260fb8
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 05d57be80a4120949fc40799d023eeab04447f91
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java 1776d8df4310b0f43edc1df4546ce6839548c127
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java b18ef712879362fad3aa9403b103bec87fbe2522
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java 14d68f386c1f673135b35185cb03bcf7d0cc1a74
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java 09b7ebed255e28b387763cf9e70c0a621e936cb9
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java 89d909dfe735321d15d8c39dcdc97a02ae97c818
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java ca38d9c1963f366b558750377130c72045709031
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java 426aa3a70397ca409557f13260c9a26b3d0a2bbb
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java 2da96d770dd4212d13390fced0bdab21720326dd
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java dd7d00636bbe2fec267d586165814a14552e17e6
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java f4e61806bc1e6b75b6114c4709021bfb8b7e29b9
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java f74c66f6d75c65075ba2e209bc138462c9a26b36
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java PRE-CREATION
exec/java-exec/src/test/resources/drill-module.conf 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
exec/java-exec/src/test/resources/drill-spool-test-module.conf PRE-CREATION
exec/java-exec/src/test/resources/work/batch/multiple_exchange.json PRE-CREATION
Diff: https://reviews.apache.org/r/15564/diff/
Testing
-------
Thanks,
Steven Phillips
Re: Review Request 15564: Patch for DRILL-274
Posted by Steven Phillips <sp...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15564/
-----------------------------------------------------------
(Updated Nov. 19, 2013, 1:12 a.m.)
Review request for drill.
Bugs: DRILL-274
https://issues.apache.org/jira/browse/DRILL-274
Repository: drill-git
Description (updated)
-------
DRILL-274: resolve merge conflict
DRILL-274: Address code review comments.
DRILL-274: Spooling batch buffer
Diffs (updated)
-----
distribution/src/resources/drill-override.conf 18a2a6a80784951ab8c78b58698f65f0575aa783
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 674dafca0c8b7840f4af0edf5b34741b3a06027d
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WireRecordBatch.java 0b0214ac610e949a0d23ff16f3f5f0756a330f77
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java 6d8a28424e3248c21876a1e1c5600ea5855c681b
exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatchProvider.java 3390af9c354c5b5d4fc56727bc7723c9bb3c6e07
exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BatchCollector.java 236b2398f36d6e22ae83f6be0095ef8d64186be7
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/RawBatchBuffer.java 82ed1ca1f17f9be93f7cb1863000b3e564417222
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java a78ffc310506c067b0139e24cb5683adf8549324
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java 3ec94929d708f4d5f02d657162369f3dbf1920c6
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java 54bf0fd29651bb179af872c0dad436f43e260fb8
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 05d57be80a4120949fc40799d023eeab04447f91
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java 1776d8df4310b0f43edc1df4546ce6839548c127
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java b18ef712879362fad3aa9403b103bec87fbe2522
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java 14d68f386c1f673135b35185cb03bcf7d0cc1a74
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java 09b7ebed255e28b387763cf9e70c0a621e936cb9
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java 89d909dfe735321d15d8c39dcdc97a02ae97c818
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java ca38d9c1963f366b558750377130c72045709031
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java 426aa3a70397ca409557f13260c9a26b3d0a2bbb
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java 2da96d770dd4212d13390fced0bdab21720326dd
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java dd7d00636bbe2fec267d586165814a14552e17e6
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java f4e61806bc1e6b75b6114c4709021bfb8b7e29b9
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java f74c66f6d75c65075ba2e209bc138462c9a26b36
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java PRE-CREATION
exec/java-exec/src/test/resources/drill-module.conf 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
exec/java-exec/src/test/resources/drill-spool-test-module.conf PRE-CREATION
exec/java-exec/src/test/resources/work/batch/multiple_exchange.json PRE-CREATION
Diff: https://reviews.apache.org/r/15564/diff/
Testing
-------
Thanks,
Steven Phillips
Re: Review Request 15564: Patch for DRILL-274
Posted by Steven Phillips <sp...@maprtech.com>.
See inline.
On Fri, Nov 15, 2013 at 10:21 PM, Timothy Chen <tn...@gmail.com> wrote:
> Sorry got no internet access on my laptop so have to put comments here in
> email:
>
> 1, from the rawspoolbuffer class I see it's intended for multithreaded
> with volatiles, but I have question when multiple threads get from the
> buffer. Looks like when the buffer is empty and is unfinished, it blocks
> until it finds an item or a interrupt exception happens. I wonder if it is
> marked finished while it's blocked? Does it hang forever since there is no
> timeout on the take?
>
>
The SpoolingRawBatchBuffer is an implementation of the RawBatchBuffer
interface. For a while now, Drill has been using UnlimitedRawBatchBuffer.
In both implementations, this is handled by interrupting the thread and
catching the exception.
> 2, the spooling test doesn't seem to test spooling as I think it will pass
> even without spooling given enough direct memory right? Perhaps should
> verify the spooled data is same as expected?
I have a special configuration file that I use for this test which limits
the number of execution threads to 1. The physical plan has exchanges, so
that ensures that there are multiple fragments, and since there is only one
thread, it forces it to spool.
>
Tim
>
> Sent from my iPhone
>
> > On Nov 14, 2013, at 6:00 PM, "Steven Phillips" <sp...@maprtech.com>
> wrote:
> >
> >
> > -----------------------------------------------------------
> > This is an automatically generated e-mail. To reply, visit:
> > https://reviews.apache.org/r/15564/
> > -----------------------------------------------------------
> >
> > Review request for drill.
> >
> >
> > Bugs: DRILL-274
> > https://issues.apache.org/jira/browse/DRILL-274
> >
> >
> > Repository: drill-git
> >
> >
> > Description
> > -------
> >
> > DRILL-274: Spooling batch buffer
> >
> >
> > Diffs
> > -----
> >
> > distribution/src/resources/drill-override.conf
> 7694cedf62ae5dde1221d3bef0ac6cdaedaf78d4
> > exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
> 36504f6df36d3c3bf7bf6b158f777777125fef74
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
> a7f6d2ef03eb2fef20fed5f327a12ea3c82989de
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java
> PRE-CREATION
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java
> d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java
> 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java
> c9e56085aeebfeb44b2ae82c981251a7d3e47675
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java
> 670347cba07b4c5010053a89a6352973d40a5dca
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java
> af127781bb1d6bfab312d2843a3d41eadbf9f4f4
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
> PRE-CREATION
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java
> 43870da545a4659f85bf608c4d2fda1f808765cf
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java
> 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
> >
> exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java
> d947d683d4686cabf30b85b927ed8fe3847ef6f1
> > exec/java-exec/src/main/resources/drill-module.conf
> 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java
> ac05319725e0bf797a67a133f44a9a6988ab12c2
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java
> c47fa89ab65b18ca21fd4b2f2dc9d478b5fe4809
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java
> 7e7ac633aacd70469cbf83f4b52641853d690f1f
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java
> cbb3e5495fdf0a3efc093646c461b9216e2bc486
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java
> 7ce69e2bf7248aa78444d89524c3d4d06e275835
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java
> e81774ad74f2eeeeb78e68216c5ab0a3b60f6219
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java
> ceebe559f5f9d605aab13a1a204443249c012292
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java
> ef7638ede343b055a6686ffaa7f7d6d72a97c118
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java
> 884887d7de1542974766185b3c9675af7dcfccbd
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java
> fd813b41224b309669ffdbe6c1bd1662458fe3b3
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java
> 634182a57a1533d8410b086c4e4b7ca18a23041a
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java
> ea14b08d6267e2fb1b1e950d089d94320edac3f1
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java
> 4c04abcd38c5d77f0b8da7f263ed220c0a568ddf
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java
> 92c745fa08cae70b5ed991b87ce751613f0cbfdc
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java
> PRE-CREATION
> >
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java
> PRE-CREATION
> > exec/java-exec/src/test/resources/drill-module.conf
> 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
> > exec/java-exec/src/test/resources/drill-spool-test-module.conf
> PRE-CREATION
> > exec/java-exec/src/test/resources/work/batch/multiple_exchange.json
> PRE-CREATION
> >
> > Diff: https://reviews.apache.org/r/15564/diff/
> >
> >
> > Testing
> > -------
> >
> >
> > Thanks,
> >
> > Steven Phillips
> >
>
Re: Review Request 15564: Patch for DRILL-274
Posted by Timothy Chen <tn...@gmail.com>.
Sorry got no internet access on my laptop so have to put comments here in email:
1, from the rawspoolbuffer class I see it's intended for multithreaded with volatiles, but I have question when multiple threads get from the buffer. Looks like when the buffer is empty and is unfinished, it blocks until it finds an item or a interrupt exception happens. I wonder if it is marked finished while it's blocked? Does it hang forever since there is no timeout on the take?
2, the spooling test doesn't seem to test spooling as I think it will pass even without spooling given enough direct memory right? Perhaps should verify the spooled data is same as expected?
Tim
Sent from my iPhone
> On Nov 14, 2013, at 6:00 PM, "Steven Phillips" <sp...@maprtech.com> wrote:
>
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15564/
> -----------------------------------------------------------
>
> Review request for drill.
>
>
> Bugs: DRILL-274
> https://issues.apache.org/jira/browse/DRILL-274
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-274: Spooling batch buffer
>
>
> Diffs
> -----
>
> distribution/src/resources/drill-override.conf 7694cedf62ae5dde1221d3bef0ac6cdaedaf78d4
> exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
> exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java a7f6d2ef03eb2fef20fed5f327a12ea3c82989de
> exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
> exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
> exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
> exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java ac05319725e0bf797a67a133f44a9a6988ab12c2
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java c47fa89ab65b18ca21fd4b2f2dc9d478b5fe4809
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 7e7ac633aacd70469cbf83f4b52641853d690f1f
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java cbb3e5495fdf0a3efc093646c461b9216e2bc486
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java 7ce69e2bf7248aa78444d89524c3d4d06e275835
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java e81774ad74f2eeeeb78e68216c5ab0a3b60f6219
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java ceebe559f5f9d605aab13a1a204443249c012292
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java ef7638ede343b055a6686ffaa7f7d6d72a97c118
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java 884887d7de1542974766185b3c9675af7dcfccbd
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java fd813b41224b309669ffdbe6c1bd1662458fe3b3
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java 634182a57a1533d8410b086c4e4b7ca18a23041a
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java ea14b08d6267e2fb1b1e950d089d94320edac3f1
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java 4c04abcd38c5d77f0b8da7f263ed220c0a568ddf
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java 92c745fa08cae70b5ed991b87ce751613f0cbfdc
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/FileTest.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/work/batch/TestSpoolingBuffer.java PRE-CREATION
> exec/java-exec/src/test/resources/drill-module.conf 99dd863ec802a90c5d7b8fed4ff44651f4d65f96
> exec/java-exec/src/test/resources/drill-spool-test-module.conf PRE-CREATION
> exec/java-exec/src/test/resources/work/batch/multiple_exchange.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15564/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Steven Phillips
>