You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Christian Smith <ch...@greenbutton.com> on 2013/09/11 11:30:15 UTC
No MapReduce Jobs in UI
Hi,
I've configured an Ambari cluster with 7 nodes including HDFS, MapReduce,
Ganglia and Nagios. The UI says all services are OK (green).
Under the Jobs tab I don't see any MapReduce jobs. I've manually run
several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
--config ... job list'.
I don't need Oozie, correct? And if not, what configuration could I have
missed?
Thanks,
Christian
Re: No MapReduce Jobs in UI
Posted by Yusaku Sako <yu...@hortonworks.com>.
Cool. Glad that worked!
Yusaku
On Wed, Sep 11, 2013 at 1:03 PM, Christian Smith
<ch...@greenbutton.com>wrote:
> Hi Yusaku,
>
> That was the issue, thanks. It seems you spotted a bug in our code as all
> our booleans are being converted to invalid string equivalents.
>
> Thanks,
> Christian
>
>
> On Wed, Sep 11, 2013 at 11:04 PM, Yusaku Sako <yu...@hortonworks.com>wrote:
>
>> Ambari Web client deploys with "rca_enabled": "true" (lowercase t).
>> Can you fix that and restart MapReduce via Ambari?
>>
>> Please check the file /etc/hadoop/conf/log4j.properties on the JobTracker
>> host.
>> Towards the bottom, you should see something like the following.
>>
>> #
>> # Job Summary Appender
>> #
>> # Use following logger to send summary to separate file defined by
>> # hadoop.mapreduce.jobsummary.log.file rolled daily:
>> # hadoop.mapreduce.jobsummary.logger=INFO,JSA
>> #
>> hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
>> hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
>> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
>>
>> log4j.appender.JSA.File=${hadoop.log.dir}/${hadoop.mapreduce.jobsummary.log.file}
>> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
>> log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
>> %c{2}: %m%n
>> log4j.appender.JSA.DatePattern=.yyyy-MM-dd
>>
>> log4j.logger.org.apache.hadoop.mapred.JobInProgress$JobSummary=${hadoop.mapreduce.jobsummary.logger}
>> log4j.additivity.org.apache.hadoop.mapred.JobInProgress$JobSummary=false
>>
>>
>> ambari.jobhistory.database=jdbc:postgresql://
>> c6404.ambari.apache.org/ambarirca
>> ambari.jobhistory.driver=org.postgresql.Driver
>> ambari.jobhistory.user=mapred
>> ambari.jobhistory.password=mapred
>> ambari.jobhistory.logger=DEBUG,JHA
>>
>>
>> log4j.appender.JHA=org.apache.ambari.log4j.hadoop.mapreduce.jobhistory.JobHistoryAppender
>> log4j.appender.JHA.database=${ambari.jobhistory.database}
>> log4j.appender.JHA.driver=${ambari.jobhistory.driver}
>> log4j.appender.JHA.user=${ambari.jobhistory.user}
>> log4j.appender.JHA.password=${ambari.jobhistory.password}
>>
>>
>> log4j.logger.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=${ambari.jobhistory.logger}
>> log4j.additivity.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=true
>>
>>
>> Yusaku
>>
>> On Wed, Sep 11, 2013 at 2:49 AM, Christian Smith <
>> christian@greenbutton.com> wrote:
>>
>>> Hi Yusaku,
>>>
>>> Sorry I forgot to mention I configured the cluster via the API. Ambari
>>> version 1.2.4.
>>>
>>> From the global config (reported via the API):
>>>
>>> "properties" : { "rca_enabled" : "True",
>>>
>>>
>>> Thanks,
>>> Christian
>>>
>>>
>>> On Wed, Sep 11, 2013 at 9:41 PM, Yusaku Sako <yu...@hortonworks.com>wrote:
>>>
>>>> Hi Christian,
>>>>
>>>> Can you check the "global" configuration on the cluster?
>>>> GET http://ambari-host:8080/api/v1/clusters/<cluster>
>>>> and follow the link for the "global" one.
>>>> In the global configuration, do you see "rca_enabled" set to "true"?
>>>>
>>>> BTW, what version of Ambari are you using, and did you install the
>>>> cluster via the wizard or via the API?
>>>>
>>>> Yusaku
>>>>
>>>>
>>>> On Wed, Sep 11, 2013 at 2:30 AM, Christian Smith <
>>>> christian@greenbutton.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've configured an Ambari cluster with 7 nodes including HDFS,
>>>>> MapReduce, Ganglia and Nagios. The UI says all services are OK (green).
>>>>>
>>>>> Under the Jobs tab I don't see any MapReduce jobs. I've manually run
>>>>> several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
>>>>> --config ... job list'.
>>>>>
>>>>> I don't need Oozie, correct? And if not, what configuration could I
>>>>> have missed?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>
>>>>
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or
>>>> entity to which it is addressed and may contain information that is
>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>> If the reader of this message is not the intended recipient, you are hereby
>>>> notified that any printing, copying, dissemination, distribution,
>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>> you have received this communication in error, please contact the sender
>>>> immediately and delete it from your system. Thank You.
>>>
>>>
>>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: No MapReduce Jobs in UI
Posted by Christian Smith <ch...@greenbutton.com>.
Hi Yusaku,
That was the issue, thanks. It seems you spotted a bug in our code as all
our booleans are being converted to invalid string equivalents.
Thanks,
Christian
On Wed, Sep 11, 2013 at 11:04 PM, Yusaku Sako <yu...@hortonworks.com>wrote:
> Ambari Web client deploys with "rca_enabled": "true" (lowercase t).
> Can you fix that and restart MapReduce via Ambari?
>
> Please check the file /etc/hadoop/conf/log4j.properties on the JobTracker
> host.
> Towards the bottom, you should see something like the following.
>
> #
> # Job Summary Appender
> #
> # Use following logger to send summary to separate file defined by
> # hadoop.mapreduce.jobsummary.log.file rolled daily:
> # hadoop.mapreduce.jobsummary.logger=INFO,JSA
> #
> hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
> hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
>
> log4j.appender.JSA.File=${hadoop.log.dir}/${hadoop.mapreduce.jobsummary.log.file}
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
> %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.yyyy-MM-dd
>
> log4j.logger.org.apache.hadoop.mapred.JobInProgress$JobSummary=${hadoop.mapreduce.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapred.JobInProgress$JobSummary=false
>
>
> ambari.jobhistory.database=jdbc:postgresql://
> c6404.ambari.apache.org/ambarirca
> ambari.jobhistory.driver=org.postgresql.Driver
> ambari.jobhistory.user=mapred
> ambari.jobhistory.password=mapred
> ambari.jobhistory.logger=DEBUG,JHA
>
>
> log4j.appender.JHA=org.apache.ambari.log4j.hadoop.mapreduce.jobhistory.JobHistoryAppender
> log4j.appender.JHA.database=${ambari.jobhistory.database}
> log4j.appender.JHA.driver=${ambari.jobhistory.driver}
> log4j.appender.JHA.user=${ambari.jobhistory.user}
> log4j.appender.JHA.password=${ambari.jobhistory.password}
>
>
> log4j.logger.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=${ambari.jobhistory.logger}
> log4j.additivity.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=true
>
>
> Yusaku
>
> On Wed, Sep 11, 2013 at 2:49 AM, Christian Smith <
> christian@greenbutton.com> wrote:
>
>> Hi Yusaku,
>>
>> Sorry I forgot to mention I configured the cluster via the API. Ambari
>> version 1.2.4.
>>
>> From the global config (reported via the API):
>>
>> "properties" : { "rca_enabled" : "True",
>>
>>
>> Thanks,
>> Christian
>>
>>
>> On Wed, Sep 11, 2013 at 9:41 PM, Yusaku Sako <yu...@hortonworks.com>wrote:
>>
>>> Hi Christian,
>>>
>>> Can you check the "global" configuration on the cluster?
>>> GET http://ambari-host:8080/api/v1/clusters/<cluster>
>>> and follow the link for the "global" one.
>>> In the global configuration, do you see "rca_enabled" set to "true"?
>>>
>>> BTW, what version of Ambari are you using, and did you install the
>>> cluster via the wizard or via the API?
>>>
>>> Yusaku
>>>
>>>
>>> On Wed, Sep 11, 2013 at 2:30 AM, Christian Smith <
>>> christian@greenbutton.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've configured an Ambari cluster with 7 nodes including HDFS,
>>>> MapReduce, Ganglia and Nagios. The UI says all services are OK (green).
>>>>
>>>> Under the Jobs tab I don't see any MapReduce jobs. I've manually run
>>>> several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
>>>> --config ... job list'.
>>>>
>>>> I don't need Oozie, correct? And if not, what configuration could I
>>>> have missed?
>>>>
>>>> Thanks,
>>>> Christian
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: No MapReduce Jobs in UI
Posted by Yusaku Sako <yu...@hortonworks.com>.
Ambari Web client deploys with "rca_enabled": "true" (lowercase t).
Can you fix that and restart MapReduce via Ambari?
Please check the file /etc/hadoop/conf/log4j.properties on the JobTracker
host.
Towards the bottom, you should see something like the following.
#
# Job Summary Appender
#
# Use following logger to send summary to separate file defined by
# hadoop.mapreduce.jobsummary.log.file rolled daily:
# hadoop.mapreduce.jobsummary.logger=INFO,JSA
#
hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.JSA.File=${hadoop.log.dir}/${hadoop.mapreduce.jobsummary.log.file}
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}:
%m%n
log4j.appender.JSA.DatePattern=.yyyy-MM-dd
log4j.logger.org.apache.hadoop.mapred.JobInProgress$JobSummary=${hadoop.mapreduce.jobsummary.logger}
log4j.additivity.org.apache.hadoop.mapred.JobInProgress$JobSummary=false
ambari.jobhistory.database=jdbc:postgresql://
c6404.ambari.apache.org/ambarirca
ambari.jobhistory.driver=org.postgresql.Driver
ambari.jobhistory.user=mapred
ambari.jobhistory.password=mapred
ambari.jobhistory.logger=DEBUG,JHA
log4j.appender.JHA=org.apache.ambari.log4j.hadoop.mapreduce.jobhistory.JobHistoryAppender
log4j.appender.JHA.database=${ambari.jobhistory.database}
log4j.appender.JHA.driver=${ambari.jobhistory.driver}
log4j.appender.JHA.user=${ambari.jobhistory.user}
log4j.appender.JHA.password=${ambari.jobhistory.password}
log4j.logger.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=${ambari.jobhistory.logger}
log4j.additivity.org.apache.hadoop.mapred.JobHistory$JobHistoryLogger=true
Yusaku
On Wed, Sep 11, 2013 at 2:49 AM, Christian Smith
<ch...@greenbutton.com>wrote:
> Hi Yusaku,
>
> Sorry I forgot to mention I configured the cluster via the API. Ambari
> version 1.2.4.
>
> From the global config (reported via the API):
>
> "properties" : { "rca_enabled" : "True",
>
>
> Thanks,
> Christian
>
>
> On Wed, Sep 11, 2013 at 9:41 PM, Yusaku Sako <yu...@hortonworks.com>wrote:
>
>> Hi Christian,
>>
>> Can you check the "global" configuration on the cluster?
>> GET http://ambari-host:8080/api/v1/clusters/<cluster>
>> and follow the link for the "global" one.
>> In the global configuration, do you see "rca_enabled" set to "true"?
>>
>> BTW, what version of Ambari are you using, and did you install the
>> cluster via the wizard or via the API?
>>
>> Yusaku
>>
>>
>> On Wed, Sep 11, 2013 at 2:30 AM, Christian Smith <
>> christian@greenbutton.com> wrote:
>>
>>> Hi,
>>>
>>> I've configured an Ambari cluster with 7 nodes including HDFS,
>>> MapReduce, Ganglia and Nagios. The UI says all services are OK (green).
>>>
>>> Under the Jobs tab I don't see any MapReduce jobs. I've manually run
>>> several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
>>> --config ... job list'.
>>>
>>> I don't need Oozie, correct? And if not, what configuration could I
>>> have missed?
>>>
>>> Thanks,
>>> Christian
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: No MapReduce Jobs in UI
Posted by Christian Smith <ch...@greenbutton.com>.
Hi Yusaku,
Sorry I forgot to mention I configured the cluster via the API. Ambari
version 1.2.4.
>From the global config (reported via the API):
"properties" : { "rca_enabled" : "True",
Thanks,
Christian
On Wed, Sep 11, 2013 at 9:41 PM, Yusaku Sako <yu...@hortonworks.com> wrote:
> Hi Christian,
>
> Can you check the "global" configuration on the cluster?
> GET http://ambari-host:8080/api/v1/clusters/<cluster>
> and follow the link for the "global" one.
> In the global configuration, do you see "rca_enabled" set to "true"?
>
> BTW, what version of Ambari are you using, and did you install the cluster
> via the wizard or via the API?
>
> Yusaku
>
>
> On Wed, Sep 11, 2013 at 2:30 AM, Christian Smith <
> christian@greenbutton.com> wrote:
>
>> Hi,
>>
>> I've configured an Ambari cluster with 7 nodes including HDFS,
>> MapReduce, Ganglia and Nagios. The UI says all services are OK (green).
>>
>> Under the Jobs tab I don't see any MapReduce jobs. I've manually run
>> several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
>> --config ... job list'.
>>
>> I don't need Oozie, correct? And if not, what configuration could I have
>> missed?
>>
>> Thanks,
>> Christian
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: No MapReduce Jobs in UI
Posted by Yusaku Sako <yu...@hortonworks.com>.
Hi Christian,
Can you check the "global" configuration on the cluster?
GET http://ambari-host:8080/api/v1/clusters/<cluster>
and follow the link for the "global" one.
In the global configuration, do you see "rca_enabled" set to "true"?
BTW, what version of Ambari are you using, and did you install the cluster
via the wizard or via the API?
Yusaku
On Wed, Sep 11, 2013 at 2:30 AM, Christian Smith
<ch...@greenbutton.com>wrote:
> Hi,
>
> I've configured an Ambari cluster with 7 nodes including HDFS, MapReduce,
> Ganglia and Nagios. The UI says all services are OK (green).
>
> Under the Jobs tab I don't see any MapReduce jobs. I've manually run
> several against the jobtracker which show up if I do a 'sudo -u hdfs hadoop
> --config ... job list'.
>
> I don't need Oozie, correct? And if not, what configuration could I have
> missed?
>
> Thanks,
> Christian
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.