You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by abdelhakim deneche <ad...@gmail.com> on 2015/05/13 19:53:09 UTC
Review Request 34173: DRILL-3053: add unchecked exception injection
site in ChildAllocator
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------
Review request for drill, Chris Westin and Jacques Nadeau.
Bugs: DRILL-3053
https://issues.apache.org/jira/browse/DRILL-3053
Repository: drill-git
Description
-------
added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
Diffs
-----
common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
Diff: https://reviews.apache.org/r/34173/diff/
Testing
-------
pending results from unit tests / cluster
Thanks,
abdelhakim deneche
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83670
-----------------------------------------------------------
Ship it!
One small change. Otherwise, ship it.
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134681>
need final
- Jacques Nadeau
On May 13, 2015, 9:36 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 9:36 p.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> unit tests are passing...
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83673
-----------------------------------------------------------
Ship it (non-binding)
- Chris Westin
On May 13, 2015, 3:05 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 3:05 p.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> testing...
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------
(Updated May 13, 2015, 10:05 p.m.)
Review request for drill, Chris Westin and Jacques Nadeau.
Changes
-------
small code fixes
Bugs: DRILL-3053
https://issues.apache.org/jira/browse/DRILL-3053
Repository: drill-git
Description
-------
added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
Diff: https://reviews.apache.org/r/34173/diff/
Testing (updated)
-------
testing...
Thanks,
abdelhakim deneche
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------
(Updated May 13, 2015, 9:36 p.m.)
Review request for drill, Chris Westin and Jacques Nadeau.
Bugs: DRILL-3053
https://issues.apache.org/jira/browse/DRILL-3053
Repository: drill-git
Description
-------
added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
Diffs
-----
common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
Diff: https://reviews.apache.org/r/34173/diff/
Testing (updated)
-------
unit tests are passing...
Thanks,
abdelhakim deneche
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83665
-----------------------------------------------------------
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
<https://reviews.apache.org/r/34173/#comment134675>
Rename to injectUnchecked(...)
- Sudheesh Katkam
On May 13, 2015, 9:07 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 9:07 p.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> pending results from unit tests / cluster
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------
(Updated May 13, 2015, 9:07 p.m.)
Review request for drill, Chris Westin and Jacques Nadeau.
Changes
-------
addressed review comments:
- added comments to ChildAllocator.buffer() to explain how injecting a NullPointerException will force the allocator to return null instead
- made the test class smaller, this is mainly meant as an example of how this exceptions should be injected.
Bugs: DRILL-3053
https://issues.apache.org/jira/browse/DRILL-3053
Repository: drill-git
Description
-------
added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
Diff: https://reviews.apache.org/r/34173/diff/
Testing
-------
pending results from unit tests / cluster
Thanks,
abdelhakim deneche
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
> On May 13, 2015, 8:09 p.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java, line 43
> > <https://reviews.apache.org/r/34173/diff/2/?file=958274#file958274line43>
> >
> > Could this method be put into BaseTestQuery so that other tests could use it? Maybe it should be TestWithOutOfMemoryException then.
I think this test is too specific, for example it's only injecting the exception in the 1st drillbit.
- abdelhakim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------
On May 13, 2015, 5:54 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 5:54 p.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> pending results from unit tests / cluster
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
> On May 13, 2015, 8:09 p.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java, line 53
> > <https://reviews.apache.org/r/34173/diff/2/?file=958274#file958274line53>
> >
> > I might parameterize the nSkip value, so that these queries could be tested with this exception being thrown in different places.
> >
> > For example, we might define a fixed set of numbers like
> >
> > final int nSkips[] = {200, 250, 300, 375};
> >
> > testWithException takes a single number as an additional argument.
> >
> > testWithExeptions might take an array, and call testWithException in a loop with each of the values.
> >
> > Then we could test each query with failures at different locations. As above, these might be useful in BaseTestQuery, but if that's too hard for now, just do it here.
Tried doing that, but then we'll need to collect the errors and only fail the test at the end, otherwise the test won't go through all nSkip values.
I decided to make the test class smaller, so it can serve as an example of how this exception should be injected. We can later extend the tests to track allocation failures that are not handled correctly
- abdelhakim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------
On May 13, 2015, 9:07 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 9:07 p.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> pending results from unit tests / cluster
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------
A couple minor things, and a suggestion on how to get even more out of this test by testing the same queries with failures at multiple different locations.
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134664>
There could be more than one injection site, so this should have a more specific name.
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134665>
It took me a minute to understand this; it is an unusual way to use the exception injector, so please add a comment explaining what's going on here so others that aren't as familiar with it or the allocator (they might have assumed this should be throwing OutOfMemoryException) will understand why you catch the exception and return null.
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134666>
"Exception{" => "Exception {" throughout the file.
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134667>
Could this method be put into BaseTestQuery so that other tests could use it? Maybe it should be TestWithOutOfMemoryException then.
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134668>
I might parameterize the nSkip value, so that these queries could be tested with this exception being thrown in different places.
For example, we might define a fixed set of numbers like
final int nSkips[] = {200, 250, 300, 375};
testWithException takes a single number as an additional argument.
testWithExeptions might take an array, and call testWithException in a loop with each of the values.
Then we could test each query with failures at different locations. As above, these might be useful in BaseTestQuery, but if that's too hard for now, just do it here.
- Chris Westin
On May 13, 2015, 10:54 a.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 10:54 a.m.)
>
>
> Review request for drill, Chris Westin and Jacques Nadeau.
>
>
> Bugs: DRILL-3053
> https://issues.apache.org/jira/browse/DRILL-3053
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
> common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
> exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
> exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34173/diff/
>
>
> Testing
> -------
>
> pending results from unit tests / cluster
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 34173: DRILL-3053: add unchecked exception
injection site in ChildAllocator
Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------
(Updated May 13, 2015, 5:54 p.m.)
Review request for drill, Chris Westin and Jacques Nadeau.
Changes
-------
cleaned the change in ExecutionControlsInjector
Bugs: DRILL-3053
https://issues.apache.org/jira/browse/DRILL-3053
Repository: drill-git
Description
-------
added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065
common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e
exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300
exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION
Diff: https://reviews.apache.org/r/34173/diff/
Testing
-------
pending results from unit tests / cluster
Thanks,
abdelhakim deneche