You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2008/01/04 03:02:24 UTC
[Lucene-hadoop Wiki] Update of "HadoopStreaming" by TedDunning
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by TedDunning:
http://wiki.apache.org/lucene-hadoop/HadoopStreaming
------------------------------------------------------------------------------
Usage: $HADOOP_HOME/bin/hadoop jar build/hadoop-streaming.jar [options]
Options:
- -input <path> DFS input file(s) for the Map step
+ -input <path> DFS input file(s) for the Map step
- -output <path> DFS output directory for the Reduce step
+ -output <path> DFS output directory for the Reduce step
- -mapper <cmd> The streaming command to run
+ -mapper <cmd|JavaClassName> The streaming command to run
- -combiner <cmd> Not implemented. But you can pipe the mapper output
+ -combiner <JavaClassName> Combiner has to be a Java class
- -reducer <cmd> The streaming command to run
+ -reducer <cmd|JavaClassName> The streaming command to run
- -file <file> File/dir to be shipped in the Job jar file
+ -file <file> File/dir to be shipped in the Job jar file
- -cluster <name> Default uses hadoop-default.xml and hadoop-site.xml
- -config <file> Optional. One or more paths to xml config files
- -dfs <h:p> Optional. Override DFS configuration
+ -dfs <h:p>|local Optional. Override DFS configuration
- -jt <h:p> Optional. Override JobTracker configuration
+ -jt <h:p>|local Optional. Override JobTracker configuration
+ -additionalconfspec specfile Optional.
+ -inputformat TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName Optional.
+ -outputformat TextOutputFormat(default)|JavaClassName Optional.
+ -partitioner JavaClassName Optional.
+ -numReduceTasks <num> Optional.
- -inputreader <spec> Optional.
+ -inputreader <spec> Optional.
- -jobconf <n>=<v> Optional.
+ -jobconf <n>=<v> Optional. Add or override a JobConf property
- -cmdenv <n>=<v> Optional. Pass env.var to streaming commands
+ -cmdenv <n>=<v> Optional. Pass env.var to streaming commands
+ -cacheFile fileNameURI
+ -cacheArchive fileNameURI
-verbose
In -input: globbing on <path> is supported and can have multiple -input