You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Peeyush Bishnoi (JIRA)" <ji...@apache.org> on 2009/10/06 13:09:31 UTC
[jira] Created: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
-----------------------------------------------------------------------------------------------
Key: MAPREDUCE-1068
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Peeyush Bishnoi
Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
For example:
With hadoop-0.20.0:
$ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
Streaming Job Failed!
With hadoop-0.18.*:
$ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Iyappan Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776375#action_12776375 ]
Iyappan Srinivasan commented on MAPREDUCE-1068:
-----------------------------------------------
+1 for QA
bin/hadoop jar contrib/streaming/hadoop-streaming.jar -input /user/test1.txt -output /user/test.out -mapper "/bin/cat"
-file dummy -reducer "/bin/cat"
Before patch the Streaming Failed msg only came.
After patch, it gives,
File: <path>/dummy does not exist, or is not readable.
Streaming Job Failed!
So, failed log file comes properly.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Attachment: patch-1068.txt
Patch printing the error message.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774386#action_12774386 ]
Hadoop QA commented on MAPREDUCE-1068:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423898/patch-1068-1.txt
against trunk revision 833275.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/229/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/229/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/229/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/229/console
This message is automatically generated.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772702#action_12772702 ]
Hadoop QA commented on MAPREDUCE-1068:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423818/patch-1068.txt
against trunk revision 831816.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/218/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/218/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/218/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/218/console
This message is automatically generated.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1068:
--------------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.22.0)
(was: 0.20.1)
(was: 0.21.0)
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks Amareshwari.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Status: Open (was: Patch Available)
Re-submitting for hudson
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777033#action_12777033 ]
Hudson commented on MAPREDUCE-1068:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #141 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/141/])
. Fix streaming job to show proper message if file is is not present. Contributed by Amareshwari Sriramadasu.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Status: Open (was: Patch Available)
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Release Note: Fix streaming job to show proper message if file is is not present, for -file option.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776859#action_12776859 ]
Hudson commented on MAPREDUCE-1068:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #116 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/116/])
. Fix streaming job to show proper message if file is is not present. Contributed by Amareshwari Sriramadasu.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772593#action_12772593 ]
Todd Lipcon commented on MAPREDUCE-1068:
----------------------------------------
I'm not sure this is the right fix. Would it not be better to print out any exception caught at line 128? It currently says '//ignore, since log will already be printed' but this is clearly not true. In working on MAPREDUCE-967 I found that changing that to LOG.fatal("Error in stream job", ex) printed errors as expected.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774853#action_12774853 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1068:
----------------------------------------------------
bq. -1 contrib tests.
Is due to MAPREDUCE-1124
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Attachment: patch-1068-1.txt
Printing the exception log in debug mode.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772916#action_12772916 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1068:
----------------------------------------------------
bq. I'm not sure this is the right fix. Would it not be better to print out any exception caught at line 128?
fail method in streaming is used to fail the job whenever there is an invalid argument, in which it prints the appropriate message. Users might not want to see an exception trace if there is an invalid argument.
bq. In working on MAPREDUCE-967 I found that changing that to LOG.fatal("Error in stream job", ex) printed errors as expected.
Sorry, I did not understand how MR-967 is related to command-line parsing errors in Streaming.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772918#action_12772918 ]
Todd Lipcon commented on MAPREDUCE-1068:
----------------------------------------
bq. Users might not want to see an exception trace if there is an invalid argument.
Fair enough for invalid arguments, though I think at least a debug level log with the full stack trace would be helpful for unit tests.
bq. Sorry, I did not understand how MR-967 is related to command-line parsing errors in Streaming.
It's not - I had to add a unit test for streaming there because the original patch introduced a regression that didn't already have a test. In working on the streaming unit test, I found that exceptions thrown inside StreamJob never produced a stack trace in the test logs, making it pretty difficult to debug.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Status: Patch Available (was: Open)
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Status: Patch Available (was: Open)
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job
do not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772948#action_12772948 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1068:
----------------------------------------------------
The patch applies to trunk, branch 0.21 and branch 0.20 cleanly.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
> Attachments: patch-1068-1.txt, patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do
not throw proper verbose error message if file is not present
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1068:
-----------------------------------------------
Assignee: Amareshwari Sriramadasu
Status: Patch Available (was: Open)
Manually verifies error message is printed.
> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Peeyush Bishnoi
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.2, 0.21.0, 0.22.0, 0.20.1
>
> Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file
> ---
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.