You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Sumit Kumar <k....@gmail.com> on 2014/04/18 01:25:36 UTC

Tez on hadoop 2.2.0

Hi there!

I was trying to configure Tez on a hadoop 2.2.0 cluster. I've followed
steps mentioned in [1] but not able to run the sample in vii (which is
nothing but a sleep command as mentioned below in [2]). The exception trace
is below if it helps:

14/04/17 22:46:28 ERROR security.UserGroupInformation:
PriviledgedActionException as:hadoop (auth:SIMPLE)
cause:java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
        at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
        at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1238)
        at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1234)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
        at org.apache.hadoop.mapreduce.Job.connect(Job.java:1233)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1262)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
        at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:262)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at
org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:115)
        at
org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

My configurations seem to match what [1] recommends, please see below:

[hadoop@ip-10-191-3-209 ~]$ echo $HADOOP_CLASSPATH
/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/:/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/lib/

[hadoop@ip-10-191-3-209 ~]$ cat conf/tez-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property><name>tez.lib.uris</name><value>hdfs://
10.191.3.209:9000/apps/tez-0.4.0-incubating,hdfs://10.191.3.209:9000/apps/tez-0.4.0-incubating/lib
</value></property>
</configuration>


[hadoop@ip-10-191-3-209 ~]$ grep  mapreduce.framework.name conf/*.xml
conf/mapred-site.xml:  <property><name>mapreduce.framework.name
</name><value>yarn-tez</value></property>


Any idea what i could be doing wrong?

Thanks,
-Sumit

[1] http://tez.incubator.apache.org/install.html
[2] hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar
sleep -mt 1 -rt 1 -m 1 -r 1

Re: Tez on hadoop 2.2.0

Posted by Sumit Kumar <ks...@gmail.com>.
That was it, thanks. :-)


On Thu, Apr 17, 2014 at 5:20 PM, Bikas Saha <bi...@hortonworks.com> wrote:

> Missing * at the end of the 2 paths?
>
> [hadoop@ip-10-191-3-209 ~]$ echo $HADOOP_CLASSPATH
>
> /home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/:/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/lib/
>
>
>
> *From:* ksumitus@gmail.com [mailto:ksumitus@gmail.com] *On Behalf Of *Sumit
> Kumar
> *Sent:* Thursday, April 17, 2014 4:26 PM
> *To:* user@tez.incubator.apache.org
> *Subject:* Tez on hadoop 2.2.0
>
>
>
> Hi there!
>
> I was trying to configure Tez on a hadoop 2.2.0 cluster. I've followed
> steps mentioned in [1] but not able to run the sample in vii (which is
> nothing but a sleep command as mentioned below in [2]). The exception trace
> is below if it helps:
>
> 14/04/17 22:46:28 ERROR security.UserGroupInformation:
> PriviledgedActionException as:hadoop (auth:SIMPLE)
> cause:java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
> java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
>         at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1238)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1234)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>         at org.apache.hadoop.mapreduce.Job.connect(Job.java:1233)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1262)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
>         at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:262)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
>         at
> org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:115)
>         at
> org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
>
> My configurations seem to match what [1] recommends, please see below:
>
> [hadoop@ip-10-191-3-209 ~]$ echo $HADOOP_CLASSPATH
>
> /home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/:/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/lib/
>
> [hadoop@ip-10-191-3-209 ~]$ cat conf/tez-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <configuration>
>   <property><name>tez.lib.uris</name><value>hdfs://
> 10.191.3.209:9000/apps/tez-0.4.0-incubating,hdfs://10.191.3.209:9000/apps/tez-0.4.0-incubating/lib<http://10.191.3.209:9000/apps/tez-0.4.0-incubating,hdfs:/10.191.3.209:9000/apps/tez-0.4.0-incubating/lib>
> </value></property>
> </configuration>
>
>
> [hadoop@ip-10-191-3-209 ~]$ grep  mapreduce.framework.name conf/*.xml
> conf/mapred-site.xml:  <property><name>mapreduce.framework.name
> </name><value>yarn-tez</value></property>
>
>
>
> Any idea what i could be doing wrong?
>
> Thanks,
> -Sumit
>
> [1] http://tez.incubator.apache.org/install.html
> [2] hadoop jar
> share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar
> sleep -mt 1 -rt 1 -m 1 -r 1
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

RE: Tez on hadoop 2.2.0

Posted by Bikas Saha <bi...@hortonworks.com>.
Missing * at the end of the 2 paths?

[hadoop@ip-10-191-3-209 ~]$ echo $HADOOP_CLASSPATH
/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/:/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/lib/



*From:* ksumitus@gmail.com [mailto:ksumitus@gmail.com] *On Behalf Of *Sumit
Kumar
*Sent:* Thursday, April 17, 2014 4:26 PM
*To:* user@tez.incubator.apache.org
*Subject:* Tez on hadoop 2.2.0



Hi there!

I was trying to configure Tez on a hadoop 2.2.0 cluster. I've followed
steps mentioned in [1] but not able to run the sample in vii (which is
nothing but a sleep command as mentioned below in [2]). The exception trace
is below if it helps:

14/04/17 22:46:28 ERROR security.UserGroupInformation:
PriviledgedActionException as:hadoop (auth:SIMPLE)
cause:java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
        at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
        at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1238)
        at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1234)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
        at org.apache.hadoop.mapreduce.Job.connect(Job.java:1233)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1262)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
        at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:262)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at
org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:115)
        at
org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)


My configurations seem to match what [1] recommends, please see below:

[hadoop@ip-10-191-3-209 ~]$ echo $HADOOP_CLASSPATH
/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/:/home/hadoop/incubator-tez/tez-dist/target/tez-0.4.0-incubating/tez-0.4.0-incubating/lib/

[hadoop@ip-10-191-3-209 ~]$ cat conf/tez-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property><name>tez.lib.uris</name><value>hdfs://
10.191.3.209:9000/apps/tez-0.4.0-incubating,hdfs://10.191.3.209:9000/apps/tez-0.4.0-incubating/lib<http://10.191.3.209:9000/apps/tez-0.4.0-incubating,hdfs:/10.191.3.209:9000/apps/tez-0.4.0-incubating/lib>
</value></property>
</configuration>


[hadoop@ip-10-191-3-209 ~]$ grep  mapreduce.framework.name conf/*.xml
conf/mapred-site.xml:  <property><name>mapreduce.framework.name
</name><value>yarn-tez</value></property>



Any idea what i could be doing wrong?

Thanks,
-Sumit

[1] http://tez.incubator.apache.org/install.html
[2] hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar
sleep -mt 1 -rt 1 -m 1 -r 1

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.