You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by abdelhakim deneche <ad...@gmail.com> on 2015/05/13 19:53:09 UTC

Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------

Review request for drill, Chris Westin and Jacques Nadeau.


Bugs: DRILL-3053
    https://issues.apache.org/jira/browse/DRILL-3053


Repository: drill-git


Description
-------

added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure


Diffs
-----

  common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
  common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
  exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
  exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 

Diff: https://reviews.apache.org/r/34173/diff/


Testing
-------

pending results from unit tests / cluster


Thanks,

abdelhakim deneche


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83670
-----------------------------------------------------------

Ship it!


One small change.  Otherwise, ship it.


exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134681>

    need final


- Jacques Nadeau


On May 13, 2015, 9:36 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 9:36 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> unit tests are passing...
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83673
-----------------------------------------------------------


Ship it (non-binding)

- Chris Westin


On May 13, 2015, 3:05 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:05 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> testing...
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------

(Updated May 13, 2015, 10:05 p.m.)


Review request for drill, Chris Westin and Jacques Nadeau.


Changes
-------

small code fixes


Bugs: DRILL-3053
    https://issues.apache.org/jira/browse/DRILL-3053


Repository: drill-git


Description
-------

added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure


Diffs (updated)
-----

  common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
  common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
  exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
  exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 

Diff: https://reviews.apache.org/r/34173/diff/


Testing (updated)
-------

testing...


Thanks,

abdelhakim deneche


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------

(Updated May 13, 2015, 9:36 p.m.)


Review request for drill, Chris Westin and Jacques Nadeau.


Bugs: DRILL-3053
    https://issues.apache.org/jira/browse/DRILL-3053


Repository: drill-git


Description
-------

added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure


Diffs
-----

  common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
  common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
  exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
  exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 

Diff: https://reviews.apache.org/r/34173/diff/


Testing (updated)
-------

unit tests are passing...


Thanks,

abdelhakim deneche


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83665
-----------------------------------------------------------



exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
<https://reviews.apache.org/r/34173/#comment134675>

    Rename to injectUnchecked(...)


- Sudheesh Katkam


On May 13, 2015, 9:07 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 9:07 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> pending results from unit tests / cluster
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------

(Updated May 13, 2015, 9:07 p.m.)


Review request for drill, Chris Westin and Jacques Nadeau.


Changes
-------

addressed review comments:
- added comments to ChildAllocator.buffer() to explain how injecting a NullPointerException will force the allocator to return null instead
- made the test class smaller, this is mainly meant as an example of how this exceptions should be injected.


Bugs: DRILL-3053
    https://issues.apache.org/jira/browse/DRILL-3053


Repository: drill-git


Description
-------

added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure


Diffs (updated)
-----

  common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
  common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
  exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
  exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 

Diff: https://reviews.apache.org/r/34173/diff/


Testing
-------

pending results from unit tests / cluster


Thanks,

abdelhakim deneche


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.

> On May 13, 2015, 8:09 p.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java, line 43
> > <https://reviews.apache.org/r/34173/diff/2/?file=958274#file958274line43>
> >
> >     Could this method be put into BaseTestQuery so that other tests could use it? Maybe it should be TestWithOutOfMemoryException then.

I think this test is too specific, for example it's only injecting the exception in the 1st drillbit.


- abdelhakim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------


On May 13, 2015, 5:54 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 5:54 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> pending results from unit tests / cluster
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.

> On May 13, 2015, 8:09 p.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java, line 53
> > <https://reviews.apache.org/r/34173/diff/2/?file=958274#file958274line53>
> >
> >     I might parameterize the nSkip value, so that these queries could be tested with this exception being thrown in different places.
> >     
> >     For example, we might define a fixed set of numbers like
> >     
> >     final int nSkips[] = {200, 250, 300, 375};
> >     
> >     testWithException takes a single number as an additional argument.
> >     
> >     testWithExeptions might take an array, and call testWithException in a loop with each of the values.
> >     
> >     Then we could test each query with failures at different locations. As above, these might be useful in BaseTestQuery, but if that's too hard for now, just do it here.

Tried doing that, but then we'll need to collect the errors and only fail the test at the end, otherwise the test won't go through all nSkip values.

I decided to make the test class smaller, so it can serve as an example of how this exception should be injected. We can later extend the tests to track allocation failures that are not handled correctly


- abdelhakim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------


On May 13, 2015, 9:07 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 9:07 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> pending results from unit tests / cluster
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/#review83655
-----------------------------------------------------------


A couple minor things, and a suggestion on how to get even more out of this test by testing the same queries with failures at multiple different locations.


exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134664>

    There could be more than one injection site, so this should have a more specific name.



exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java
<https://reviews.apache.org/r/34173/#comment134665>

    It took me a minute to understand this; it is an unusual way to use the exception injector, so please add a comment explaining what's going on here so others that aren't as familiar with it or the allocator (they might have assumed this should be throwing OutOfMemoryException) will understand why you catch the exception and return null.



exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134666>

    "Exception{" => "Exception {" throughout the file.



exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134667>

    Could this method be put into BaseTestQuery so that other tests could use it? Maybe it should be TestWithOutOfMemoryException then.



exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java
<https://reviews.apache.org/r/34173/#comment134668>

    I might parameterize the nSkip value, so that these queries could be tested with this exception being thrown in different places.
    
    For example, we might define a fixed set of numbers like
    
    final int nSkips[] = {200, 250, 300, 375};
    
    testWithException takes a single number as an additional argument.
    
    testWithExeptions might take an array, and call testWithException in a loop with each of the values.
    
    Then we could test each query with failures at different locations. As above, these might be useful in BaseTestQuery, but if that's too hard for now, just do it here.


- Chris Westin


On May 13, 2015, 10:54 a.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34173/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 10:54 a.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3053
>     https://issues.apache.org/jira/browse/DRILL-3053
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
> You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34173/diff/
> 
> 
> Testing
> -------
> 
> pending results from unit tests / cluster
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Re: Review Request 34173: DRILL-3053: add unchecked exception injection site in ChildAllocator

Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34173/
-----------------------------------------------------------

(Updated May 13, 2015, 5:54 p.m.)


Review request for drill, Chris Westin and Jacques Nadeau.


Changes
-------

cleaned the change in ExecutionControlsInjector


Bugs: DRILL-3053
    https://issues.apache.org/jira/browse/DRILL-3053


Repository: drill-git


Description
-------

added an unchecked injection site in ChildAllocator.buffer(int, int). It's only enabled when assertions are enabled.
You can throw any unchecked or Error exception, but if you throw a NullPointerException the buffer() method will just return "null" without throwing any exception to simulate a "normal" allocation failure


Diffs (updated)
-----

  common/src/main/java/org/apache/drill/common/exceptions/DrillRuntimeException.java abc7065 
  common/src/main/java/org/apache/drill/common/exceptions/UserException.java a67cb3f 
  exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java 9670c7e 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java 387d300 
  exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java PRE-CREATION 

Diff: https://reviews.apache.org/r/34173/diff/


Testing
-------

pending results from unit tests / cluster


Thanks,

abdelhakim deneche