You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Cheolsoo Park <ch...@cloudera.com> on 2012/10/22 08:43:02 UTC
Review Request: PIG-2973 TestStreaming test times out
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7695/
-----------------------------------------------------------
Review request for pig.
Description
-------
Streaming fails with a NullPointerException when input to the streaming binary is from a file. This is because when input is asynchronous, the exec() method is called by the FileInputThread not by the ExecutableManager thread, and the JobConf is not set in the FileInputThread.
In the attached patch, I made UdfContext (which includes JobConf) copied from ExecutableManager to FileInputThread when input is asynchronous. I decided to copy UdfContext instead of JobConf because something else besides JonConf from UdfContext might be needed in the future.
I also removed tabs and tailing whites paces from the code, so you may want to hide white space changes.
Thanks!
This addresses bug PIG-2975.
https://issues.apache.org/jira/browse/PIG-2975
Diffs
-----
src/org/apache/pig/impl/streaming/ExecutableManager.java 1899e10
test/org/apache/pig/test/TestStreaming.java c3b6237
Diff: https://reviews.apache.org/r/7695/diff/
Testing
-------
Added a new unit test case that verifies that JobConf is passed to the streaming binary via environment even when input is asynchronous: testAddJobConfToEnvironmentWithASynchInput.
ant test -Dtestcase=TestStreaming passes with both hadoop 20 and 23.
Thanks,
Cheolsoo Park
Re: Review Request: PIG-2973 TestStreaming test times out
Posted by Cheolsoo Park <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7695/
-----------------------------------------------------------
(Updated Oct. 25, 2012, 6:03 a.m.)
Review request for pig.
Changes
-------
Remove whitespace changes.
Description
-------
Streaming fails with a NullPointerException when input to the streaming binary is from a file. This is because when input is asynchronous, the exec() method is called by the FileInputThread not by the ExecutableManager thread, and the JobConf is not set in the FileInputThread.
In the attached patch, I made UdfContext (which includes JobConf) copied from ExecutableManager to FileInputThread when input is asynchronous. I decided to copy UdfContext instead of JobConf because something else besides JonConf from UdfContext might be needed in the future.
I also removed tabs and tailing whites paces from the code, so you may want to hide white space changes.
Thanks!
This addresses bug PIG-2975.
https://issues.apache.org/jira/browse/PIG-2975
Diffs (updated)
-----
src/org/apache/pig/impl/streaming/ExecutableManager.java 1899e10
test/org/apache/pig/test/TestStreaming.java c3b6237
Diff: https://reviews.apache.org/r/7695/diff/
Testing
-------
Added a new unit test case that verifies that JobConf is passed to the streaming binary via environment even when input is asynchronous: testAddJobConfToEnvironmentWithASynchInput.
ant test -Dtestcase=TestStreaming passes with both hadoop 20 and 23.
Thanks,
Cheolsoo Park