You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Istabrak Abdul-Fatah <if...@gmail.com> on 2015/08/11 15:51:13 UTC

Issues and Questions on running Hadoop 2.7 Yarn MapReduce examples

Greetings to all,
I have installed and configured Hadoop2.7.0 on a linux VM. Then I
successfully ran the pre-compiled/packaged examples (e.g. PI, WordCount
etc..).
I also downloaded the Hadoop2.7.0 source code and created an eclipse
project.
I exported the WordCount jar file and tried to run the example from the
command line as follows:

> yarn jar /opt/yarn/my_examples/WordCount.jar
/user/yarn/input/wordcount.txt output


Q1: When I used the default WordCount implementation (shown in listing1),
it failed with a list of exceptions and suggestions to use the Tool and
toolRunner interfaces/utils (see errorListing1).
      I updated the code (see listing2) and included these suggested utils
and it ran successfully.
     Could you please provide an explanation as to why the application
failed to run in the first attempt and the necessity to use the
Tool/tooRunner utils?


Q2: Does this example create a yarn client implicitly and interacts with
the Yarn layer? if not, then could you please explain how the application
interacted with the hdfs layer given that the
      Yarn later is in between?


Thx and BR

Ista