[jira] Commented: (HADOOP-1952) Streaming does not handle invalid -inputformat (typo by users for example)

["Arun C Murthy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540721 ] 

Arun C Murthy commented on HADOOP-1952:
---------------------------------------

Lohit, I see your latest patch removes some combiner options... are you sure about these changes? If so, are they related to this jira?

{noformat}
Index: src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java
===================================================================
--- src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java  (revision 588543)
+++ src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java  (working copy)
@@ -47,7 +47,6 @@
       "-input", INVALID_INPUT_FILE.getAbsolutePath(),
       "-output", OUTPUT_DIR.getAbsolutePath(),
       "-mapper", map,
-      "-combiner", combine,
       "-reducer", reduce,
       //"-verbose",
       //"-jobconf", "stream.debug=set"
Index: src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java
===================================================================
--- src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java (revision 588543)
+++ src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java (working copy)
@@ -47,7 +47,6 @@
       "-input", INPUT_FILE.getAbsolutePath(),
       "-output", OUTPUT_DIR.getAbsolutePath(),
       "-mapper", map,
-      "-combiner", combine,
       "-reducer", reduce,
       "-jobconf", "stream.recordreader.compression=gzip"
     };
Index: src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java
===================================================================
--- src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java (revision 588543)
+++ src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java (working copy)
@@ -40,7 +40,6 @@
   protected String map = StreamUtil.makeJavaCommand(TrApp.class, new String[]{".", "\\n"});
   // reduce behave like /usr/bin/uniq. But also prepend lines with R.
   // command-line combiner does not have any effect any more.
-  protected String combine  = StreamUtil.makeJavaCommand(UniqApp.class, new String[]{"C"});
   protected String reduce = StreamUtil.makeJavaCommand(UniqApp.class, new String[]{"R"});
   protected String outputExpect = "Rare\t\nRblue\t\nRbunnies\t\nRpink\t\nRred\t\nRroses\t\nRviolets\t\n";

@@ -66,7 +65,6 @@
       "-input", INPUT_FILE.getAbsolutePath(),
       "-output", OUTPUT_DIR.getAbsolutePath(),
       "-mapper", map,
-      "-combiner", combine,
       "-reducer", reduce,
       //"-verbose",
       //"-jobconf", "stream.debug=set"
{noformat}


> Streaming does not handle invalid -inputformat  (typo by users for example)
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-1952
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1952
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.14.1
>            Reporter: lohit vijayarenu
>            Assignee: lohit vijayarenu
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: CatchInvalidInputFormat.patch, HADOOP-1952-1.patch, HADOOP-1952-2.patch
>
>
> Hadoop Streaming does not handle invalid inputformat class. For example -inputformat INVALID class would not be thrown as an error. Instead it defaults to StreamInputFormat. If an invalid inputformat is specified, it is good to fail. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.