You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2008/05/13 17:16:56 UTC
[jira] Created: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Document the "stream.non.zero.exit.status.is.failure" knob for streaming
------------------------------------------------------------------------
Key: HADOOP-3379
URL: https://issues.apache.org/jira/browse/HADOOP-3379
Project: Hadoop Core
Issue Type: Improvement
Components: contrib/streaming, documentation
Affects Versions: 0.17.0
Reporter: Arun C Murthy
Priority: Minor
Fix For: 0.18.0
HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reassigned HADOOP-3379:
-----------------------------------
Assignee: Amareshwari Sriramadasu (was: Arun C Murthy)
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603830#action_12603830 ]
Devaraj Das commented on HADOOP-3379:
-------------------------------------
I think you should document the fact that people have to wrap the command in some cases as Rick pointed out.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596627#action_12596627 ]
Amareshwari Sriramadasu commented on HADOOP-3379:
-------------------------------------------------
bq. I wonder if we should just make streaming always fail the task for non-zero exit codes.
+1
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Priority: Minor
> Fix For: 0.18.0
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3379:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Incompatible change, Reviewed] (was: [Incompatible change])
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Amareshwari!
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3379:
------------------------------------
Release Note: Set default value for configuration property "stream.non.zero.exit.status.is.failure" to be "true". (was: By default, configuration property "stream.non.zero.exit.status.is.failure" is given the value "true". )
Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed, Incompatible change])
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605433#action_12605433 ]
Hudson commented on HADOOP-3379:
--------------------------------
Integrated in Hadoop-trunk #520 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/520/])
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3379:
------------------------------------
Priority: Blocker (was: Minor)
Fix Version/s: 0.18.0
Assignee: Arun C Murthy
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.18.0
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603874#action_12603874 ]
Hadoop QA commented on HADOOP-3379:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12383731/patch-3379.txt
against trunk revision 666056.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2630/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2630/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2630/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2630/console
This message is automatically generated.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-3379:
--------------------------------------------
Attachment: patch-3379.txt
Here is patch doing the following:
1. Makes default value for _stream.non.zero.exit.is.failure_ as _true_ . Thus by default, streaming tasks exiting with non-zero exit status are considered to be failed tasks.
2. Adds the documentation to streaming.xml.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604136#action_12604136 ]
Owen O'Malley commented on HADOOP-3379:
---------------------------------------
+1
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596623#action_12596623 ]
Owen O'Malley commented on HADOOP-3379:
---------------------------------------
I wonder if we should just make streaming always fail the task for non-zero exit codes.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Priority: Minor
> Fix For: 0.18.0
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Mukund Madhugiri (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mukund Madhugiri updated HADOOP-3379:
-------------------------------------
Fix Version/s: (was: 0.18.0)
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Priority: Minor
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Rick Cox (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599870#action_12599870 ]
Rick Cox commented on HADOOP-3379:
----------------------------------
It seems reasonable for the default to be true, as I think the discussion has shown users expect it.
Making it non-optional would remove the ability to run 'grep' and similar commands without a wrapper (assuming the pattern is not in every input split). Perhaps that's ok; we could just document a hint to wrap such commands in a shell script that always exits 0.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Priority: Minor
> Fix For: 0.18.0
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-3379:
--------------------------------------------
Release Note: By default, configuration property "stream.non.zero.exit.status.is.failure" is given the value "true".
Hadoop Flags: [Incompatible change]
Status: Patch Available (was: Open)
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3379) Document the
"stream.non.zero.exit.status.is.failure" knob for streaming
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603839#action_12603839 ]
Amareshwari Sriramadasu commented on HADOOP-3379:
-------------------------------------------------
Actually the patch uploaded has the configuration property _stream.non.zero.exit.is.failure_ value as _true_.
If we want 'streaming always fail the task for non-zero exit codes', we can just remove the configuration property and document to wrap commands as Rick pointed. But doesn't this make applications incompatible from 0.17 to 0.18 ? Shouldnt we deprecate the configuration property for 0.18?
Thoughts?
I think making the configuration property _stream.non.zero.exit.is.failure_ value as _true_ makes sense.
> Document the "stream.non.zero.exit.status.is.failure" knob for streaming
> ------------------------------------------------------------------------
>
> Key: HADOOP-3379
> URL: https://issues.apache.org/jira/browse/HADOOP-3379
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, documentation
> Affects Versions: 0.17.0
> Reporter: Arun C Murthy
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: patch-3379.txt
>
>
> HADOOP-2057 added a useful feature: "stream.non.zero.exit.status.is.failure" to optionally treat non-zero exit code from streaming apps as fatal. We should document this on http://hadoop.apache.org/core/docs/current/streaming.html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.