You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by "Edwards, Jonathan" <Jo...@Staples.com> on 2013/09/10 23:34:55 UTC
Ambari and Hue
Fellow Ambari Users,
I've setup a Hortonworks cluster (HDP 1.3.2) using Ambari and have HDFS, MapRed, Hive, Hbase all working nicely. I've gotten ambitious and manually installed Hue – and been able to get it working, with the exception of JobTracker integration. The reason that I'm emailing the Ambari user group is that the ultimate issue seems to be that, although properly configured in the mapped-site.xml via Ambari and present in /usr/lib/hadoop/lib on the JobTracker node managed by Ambari, the Thrift plugin isn't loading – I suspect that the jar isn't being added to the class path by the startup commands issued from the Ambari server to the remote agents. Is there a way to view what the configured class path is for the job tracker? Is there a way to override the default behavior to include additional JobTracker plugin jars?
Regards,
Jonathan Edwards
Technical Architect
Senior Systems Engineering Consultant
Staples, Inc.
1 Environmental Way | Broomfield, CO 80021
Re: Ambari and Hue
Posted by "Edwards, Jonathan" <Jo...@Staples.com>.
Michael,
Thanks for pointing this out – altering the property name as you indicated fixed everything. This is the second documentation error I've come across – the other was with the Hadoop temporary directory modifications for concurrent job submission:
9.2.e [Optional]
Originally it is written as:
<property>
<name>hadoop.tmp.dir</name>
<value>/tmp/hadoop-$USER_NAME$HUE_SUFFIX</value>
</property>
The proper values are:
<property>
<name>hadoop.tmp.dir</name>
<value>/tmp/hadoop-${user.name}${hue.suffix}</value>
</property>
Jonathan Edwards
Technical Architect
Senior Systems Engineering Consultant
Staples, Inc.
1 Environmental Way | Broomfield, CO 80021
Phone: 303.664.3256 (Desk) | 303.903.6240 (Cell) | E-Mail: Jonathan.Edwards@Staples.com<ma...@cexp.com> | Blog: joedward<http://connections.staples.com/blogs/joedward/> | Wiki:JonathanEdwards<http://wiki.cexp.com:7001/twiki/bin/view/Main/JonathanEdwards> | IM: joedward@im.staples.com<ma...@im.cexp.com>
The information contained in this email transmission and any attachments is intended only for the personal and confidential use of the designated named herein. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document and its attachment in error, and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender and return and delete the original message immediately. Thank you.
[cid:image001.jpg@01C99348.6D55AEB0]
From: Michael Harp <mi...@mikeharp.com>>
Reply-To: "ambari-user@incubator.apache.org<ma...@incubator.apache.org>" <am...@incubator.apache.org>>
Date: Tuesday, September 10, 2013 6:01 PM
To: "ambari-user@incubator.apache.org<ma...@incubator.apache.org>" <am...@incubator.apache.org>>
Subject: Re: Ambari and Hue
I hit this issue as well, turned out to be a typo in the HortonWorks instructions.
<name>mapred.jobtracker.plugins</name>
<value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
Should be:
<name>mapreduce.jobtracker.plugins</name>
<value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
Mike
On Tue, Sep 10, 2013 at 2:34 PM, Edwards, Jonathan <Jo...@staples.com>> wrote:
Fellow Ambari Users,
I've setup a Hortonworks cluster (HDP 1.3.2) using Ambari and have HDFS, MapRed, Hive, Hbase all working nicely. I've gotten ambitious and manually installed Hue – and been able to get it working, with the exception of JobTracker integration. The reason that I'm emailing the Ambari user group is that the ultimate issue seems to be that, although properly configured in the mapped-site.xml via Ambari and present in /usr/lib/hadoop/lib on the JobTracker node managed by Ambari, the Thrift plugin isn't loading – I suspect that the jar isn't being added to the class path by the startup commands issued from the Ambari server to the remote agents. Is there a way to view what the configured class path is for the job tracker? Is there a way to override the default behavior to include additional JobTracker plugin jars?
Regards,
Jonathan Edwards
Technical Architect
Senior Systems Engineering Consultant
Staples, Inc.
1 Environmental Way | Broomfield, CO 80021
Re: Ambari and Hue
Posted by Michael Harp <mi...@mikeharp.com>.
I hit this issue as well, turned out to be a typo in the HortonWorks
instructions.
<name>mapred.jobtracker.plugins</name>
<value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
Should be:
<name>mapreduce.jobtracker.plugins</name>
<value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
Mike
On Tue, Sep 10, 2013 at 2:34 PM, Edwards, Jonathan <
Jonathan.Edwards@staples.com> wrote:
> Fellow Ambari Users,
>
> I've setup a Hortonworks cluster (HDP 1.3.2) using Ambari and have HDFS,
> MapRed, Hive, Hbase all working nicely. I've gotten ambitious and manually
> installed Hue – and been able to get it working, with the exception of
> JobTracker integration. The reason that I'm emailing the Ambari user group
> is that the ultimate issue seems to be that, although properly configured
> in the mapped-site.xml via Ambari and present in /usr/lib/hadoop/lib on the
> JobTracker node managed by Ambari, the Thrift plugin isn't loading – I
> suspect that the jar isn't being added to the class path by the startup
> commands issued from the Ambari server to the remote agents. Is there a way
> to view what the configured class path is for the job tracker? Is there a
> way to override the default behavior to include additional JobTracker
> plugin jars?
>
> Regards,
>
> *Jonathan Edwards*
>
> Technical Architect
>
> Senior Systems Engineering Consultant
>
>
> *Staples, Inc.*
> 1 Environmental Way | Broomfield, CO 80021**
>
>
>
>
>
>
Re: Ambari and Hue
Posted by "Edwards, Jonathan" <Jo...@Staples.com>.
Can you check with the DEV? I see that several dependencies aren't satisfied with the Apache JobTracker class - that is it doesn't seem to know about the PluginDispatcher.
Sent from my iPhone
On Sep 10, 2013, at 3:45 PM, "Satish Abburi" <sa...@gmail.com>> wrote:
Hi Jonathan, we just went through this scenario recently and got this working. Here are the quick steps…if you have more questions, let me know, i will hook up with the developer who worked on this.
1. Hue communicates with JobTracker using Thrift service to get the job details.
Hue --> Thrift --> JobTracker
2. When the JobTracker starts, it will in-turn try to start the Thrift service.
3. The thrift service, to get started, requires "org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin" class. This class is taken from the Cloudera distribution. (hue-plugins-2.3.0-cdh4.3.0.jar located at /usr/lib/hadoop/lib/)
OR
We can also build this jar, in Horton by following the link:https://github.com/hortonworks/hortonworks-sandbox
4. The above said class requires "JobTrackerPlugin" class which is not available in the Horton distribution. So, need to add this class to the library.
This class can be found at
http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-320/org/apache/hadoop/mapred/JobTrackerPlugin.jav<http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-320/org/apache/hadoop/mapred/JobTrackerPlugin.java>a
Thanks,
Satish
On Tue, Sep 10, 2013 at 2:34 PM, Edwards, Jonathan <Jo...@staples.com>> wrote:
Fellow Ambari Users,
I've setup a Hortonworks cluster (HDP 1.3.2) using Ambari and have HDFS, MapRed, Hive, Hbase all working nicely. I've gotten ambitious and manually installed Hue – and been able to get it working, with the exception of JobTracker integration. The reason that I'm emailing the Ambari user group is that the ultimate issue seems to be that, although properly configured in the mapped-site.xml via Ambari and present in /usr/lib/hadoop/lib on the JobTracker node managed by Ambari, the Thrift plugin isn't loading – I suspect that the jar isn't being added to the class path by the startup commands issued from the Ambari server to the remote agents. Is there a way to view what the configured class path is for the job tracker? Is there a way to override the default behavior to include additional JobTracker plugin jars?
Regards,
Jonathan Edwards
Technical Architect
Senior Systems Engineering Consultant
Staples, Inc.
1 Environmental Way | Broomfield, CO 80021
Re: Ambari and Hue
Posted by Satish Abburi <sa...@gmail.com>.
Hi Jonathan, we just went through this scenario recently and got this
working. Here are the quick steps…if you have more questions, let me know,
i will hook up with the developer who worked on this.
1. Hue communicates with JobTracker using Thrift service to get the job
details.
Hue --> Thrift --> JobTracker
2. When the JobTracker starts, it will in-turn try to start the
Thrift service.
3. The thrift service, to get started, requires
"org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin" class. This class is
taken from the Cloudera distribution. (hue-plugins-2.3.0-cdh4.3.0.jar
located at /usr/lib/hadoop/lib/)
OR
We can also build this jar, in Horton by following
the link:https://github.com/hortonworks/hortonworks-sandbox
4. The above said class requires "JobTrackerPlugin" class which is
not available in the Horton distribution. So, need to add this class to
the library.
This class can be found at
http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-320/org/apache/hadoop/mapred/JobTrackerPlugin.jav<http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-320/org/apache/hadoop/mapred/JobTrackerPlugin.java>
a
Thanks,
Satish
On Tue, Sep 10, 2013 at 2:34 PM, Edwards, Jonathan <
Jonathan.Edwards@staples.com> wrote:
> Fellow Ambari Users,
>
> I've setup a Hortonworks cluster (HDP 1.3.2) using Ambari and have HDFS,
> MapRed, Hive, Hbase all working nicely. I've gotten ambitious and manually
> installed Hue – and been able to get it working, with the exception of
> JobTracker integration. The reason that I'm emailing the Ambari user group
> is that the ultimate issue seems to be that, although properly configured
> in the mapped-site.xml via Ambari and present in /usr/lib/hadoop/lib on the
> JobTracker node managed by Ambari, the Thrift plugin isn't loading – I
> suspect that the jar isn't being added to the class path by the startup
> commands issued from the Ambari server to the remote agents. Is there a way
> to view what the configured class path is for the job tracker? Is there a
> way to override the default behavior to include additional JobTracker
> plugin jars?
>
> Regards,
>
> *Jonathan Edwards*
>
> Technical Architect
>
> Senior Systems Engineering Consultant
>
>
> *Staples, Inc.*
> 1 Environmental Way | Broomfield, CO 80021**
>
>
>
>
>
>