You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Devaraj K (Created) (JIRA)" <ji...@apache.org> on 2012/03/28 15:38:26 UTC
[jira] [Created] (MAPREDUCE-4077) Issue while using Hadoop
Streaming job
Issue while using Hadoop Streaming job
--------------------------------------
Key: MAPREDUCE-4077
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
When we use -file option it says deprecated and use -files.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/3 -mapper cat -reducer wc -file hadoop
02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.
{code}
But when we use -files option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
/test/output/3 -mapper cat -reducer wc -files hadoop
02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
When we use -archives option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
But in the options it will display the usage of the -archives.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
..........
..........
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAPREDUCE-4077) Issues while using Hadoop
Streaming job
Posted by "Devaraj K (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K reopened MAPREDUCE-4077:
----------------------------------
Reopening this issue to improve the usage info.
> Issues while using Hadoop Streaming job
> ---------------------------------------
>
> Key: MAPREDUCE-4077
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Devaraj K
> Assignee: Devaraj K
>
> When we use -file option it says deprecated and use -files.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
> -output /test/output/3 -mapper cat -reducer wc -file hadoop
> 02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.
> {code}
> But when we use -files option, it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -files hadoop
> 02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> When we use -archives option, it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> But in the options it will display the usage of the -archives.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> ..........
> ..........
> -libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
> -archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4077) Issues while using Hadoop
Streaming job
Posted by "Robert Joseph Evans (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans resolved MAPREDUCE-4077.
--------------------------------------------
Resolution: Not A Problem
The issue is with the order of your arguments. I know this is ugly, but it is difficult to fix for various backwards compatibility reasons. The -archives and -files need to come before the -input argument. They need to be at the beginning like with the -D arguments.
> Issues while using Hadoop Streaming job
> ---------------------------------------
>
> Key: MAPREDUCE-4077
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Devaraj K
> Assignee: Devaraj K
>
> When we use -file option it says deprecated and use -files.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
> -output /test/output/3 -mapper cat -reducer wc -file hadoop
> 02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.
> {code}
> But when we use -files option, it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -files hadoop
> 02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> When we use -archives option, it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> But in the options it will display the usage of the -archives.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output
> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> ..........
> ..........
> -libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
> -archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira