You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Chen, Richard" <ri...@intel.com> on 2014/02/21 04:04:56 UTC
No job shown in Hadoop resource manager web UI when running jobs in
the cluster
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
RC: RM website address inside yarn-site.xml was corrected put there.
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Vinayakumar B <vi...@huawei.com>.
Send a simple mail to user-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
FYI, http://hadoop.apache.org/mailing_lists.html
From: Suresh M03 [mailto:Suresh.M03@mphasis.com]
Sent: 24 February 2014 11:40
To: user@hadoop.apache.org
Subject: RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com<ma...@mphasis.com> and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Vinayakumar B <vi...@huawei.com>.
Send a simple mail to user-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
FYI, http://hadoop.apache.org/mailing_lists.html
From: Suresh M03 [mailto:Suresh.M03@mphasis.com]
Sent: 24 February 2014 11:40
To: user@hadoop.apache.org
Subject: RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com<ma...@mphasis.com> and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Vinayakumar B <vi...@huawei.com>.
Send a simple mail to user-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
FYI, http://hadoop.apache.org/mailing_lists.html
From: Suresh M03 [mailto:Suresh.M03@mphasis.com]
Sent: 24 February 2014 11:40
To: user@hadoop.apache.org
Subject: RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com<ma...@mphasis.com> and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Vinayakumar B <vi...@huawei.com>.
Send a simple mail to user-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
FYI, http://hadoop.apache.org/mailing_lists.html
From: Suresh M03 [mailto:Suresh.M03@mphasis.com]
Sent: 24 February 2014 11:40
To: user@hadoop.apache.org
Subject: RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com<ma...@mphasis.com> and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Suresh M03 <Su...@mphasis.com>.
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Suresh M03 <Su...@mphasis.com>.
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Suresh M03 <Su...@mphasis.com>.
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running
jobs in the cluster
Posted by Suresh M03 <Su...@mphasis.com>.
Unsubscribe me from this list
Regards,
Suresh
+91-9941188684
From: Richard Chen [mailto:cxd3000@gmail.com]
Sent: Monday, February 24, 2014 11:38 AM
To: user@hadoop.apache.org
Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com>> wrote:
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>> wrote:
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application".
Thanks for the suggestion. I do notice that.But the result is the same though.
2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>> wrote:
Dear group,
I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this:
14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs.
Please help me to solve this problem.
Thanks!!
Richard Chen
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
--
Dislike trivia, appreciate no-nonsense!
Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com and delete this mail from your records.
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com> wrote:
> Thanks for the reply. Due to company mail filtering issue, I decided to
> use my personal mail account so that I can attach files freely in the
> future.
>
> Please see my reply inline to your questions below.
>
> Richard
>
>
> On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>wrote:
>
>> Hi Richard,
>>
>> Not sure how NPE happened on you command line, but I'd like to clarify
>> something here:
>> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
>> job" is deprecated. If you want to see all kinds of applications run by
>> your YARN cluster, please use "yarn application".
>>
>> Thanks for the suggestion. I do notice that.But the result is the same
> though.
>
>
>> 2. Job history server only shows the finished mapreduce jobs. There will
>> be another application history server that shows all completed applications
>> run by YARN, but it's not available on 2.2.
>>
>> 3. ResourceManager webUI is not the job history web UI. You should check
>> your yarn-site.xml to see what's the address of the RM webUI. It will list
>> all the applications that RM remembers.
>>
>
>
>
>> - Zhijie
>>
>>
>> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>>
>>> Dear group,
>>>
>>>
>>>
>>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>>> job -list or hadoop job -list all, it throws a NPE like this:
>>>
>>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>>> deprecated. Instead, use dfs.metrics.session-id
>>>
>>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>>> processName=JobTracker, sessionId=
>>>
>>> Exception in thread "main" java.lang.NullPointerException
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>
>>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>>
>>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>>> It shows no job was running and no job finishing although I was running
>>> jobs.
>>>
>>> Please help me to solve this problem.
>>>
>>> Thanks!!
>>>
>>>
>>>
>>> Richard Chen
>>>
>>>
>>>
>>
>>
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
> Dislike trivia, appreciate no-nonsense!
>
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com> wrote:
> Thanks for the reply. Due to company mail filtering issue, I decided to
> use my personal mail account so that I can attach files freely in the
> future.
>
> Please see my reply inline to your questions below.
>
> Richard
>
>
> On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>wrote:
>
>> Hi Richard,
>>
>> Not sure how NPE happened on you command line, but I'd like to clarify
>> something here:
>> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
>> job" is deprecated. If you want to see all kinds of applications run by
>> your YARN cluster, please use "yarn application".
>>
>> Thanks for the suggestion. I do notice that.But the result is the same
> though.
>
>
>> 2. Job history server only shows the finished mapreduce jobs. There will
>> be another application history server that shows all completed applications
>> run by YARN, but it's not available on 2.2.
>>
>> 3. ResourceManager webUI is not the job history web UI. You should check
>> your yarn-site.xml to see what's the address of the RM webUI. It will list
>> all the applications that RM remembers.
>>
>
>
>
>> - Zhijie
>>
>>
>> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>>
>>> Dear group,
>>>
>>>
>>>
>>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>>> job -list or hadoop job -list all, it throws a NPE like this:
>>>
>>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>>> deprecated. Instead, use dfs.metrics.session-id
>>>
>>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>>> processName=JobTracker, sessionId=
>>>
>>> Exception in thread "main" java.lang.NullPointerException
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>
>>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>>
>>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>>> It shows no job was running and no job finishing although I was running
>>> jobs.
>>>
>>> Please help me to solve this problem.
>>>
>>> Thanks!!
>>>
>>>
>>>
>>> Richard Chen
>>>
>>>
>>>
>>
>>
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
> Dislike trivia, appreciate no-nonsense!
>
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com> wrote:
> Thanks for the reply. Due to company mail filtering issue, I decided to
> use my personal mail account so that I can attach files freely in the
> future.
>
> Please see my reply inline to your questions below.
>
> Richard
>
>
> On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>wrote:
>
>> Hi Richard,
>>
>> Not sure how NPE happened on you command line, but I'd like to clarify
>> something here:
>> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
>> job" is deprecated. If you want to see all kinds of applications run by
>> your YARN cluster, please use "yarn application".
>>
>> Thanks for the suggestion. I do notice that.But the result is the same
> though.
>
>
>> 2. Job history server only shows the finished mapreduce jobs. There will
>> be another application history server that shows all completed applications
>> run by YARN, but it's not available on 2.2.
>>
>> 3. ResourceManager webUI is not the job history web UI. You should check
>> your yarn-site.xml to see what's the address of the RM webUI. It will list
>> all the applications that RM remembers.
>>
>
>
>
>> - Zhijie
>>
>>
>> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>>
>>> Dear group,
>>>
>>>
>>>
>>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>>> job -list or hadoop job -list all, it throws a NPE like this:
>>>
>>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>>> deprecated. Instead, use dfs.metrics.session-id
>>>
>>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>>> processName=JobTracker, sessionId=
>>>
>>> Exception in thread "main" java.lang.NullPointerException
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>
>>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>>
>>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>>> It shows no job was running and no job finishing although I was running
>>> jobs.
>>>
>>> Please help me to solve this problem.
>>>
>>> Thanks!!
>>>
>>>
>>>
>>> Richard Chen
>>>
>>>
>>>
>>
>>
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
> Dislike trivia, appreciate no-nonsense!
>
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
found the problem was due to missing mapred-site.xml.
After creating it, the problem was gone.
thanks for all the answers.
On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen <cx...@gmail.com> wrote:
> Thanks for the reply. Due to company mail filtering issue, I decided to
> use my personal mail account so that I can attach files freely in the
> future.
>
> Please see my reply inline to your questions below.
>
> Richard
>
>
> On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com>wrote:
>
>> Hi Richard,
>>
>> Not sure how NPE happened on you command line, but I'd like to clarify
>> something here:
>> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
>> job" is deprecated. If you want to see all kinds of applications run by
>> your YARN cluster, please use "yarn application".
>>
>> Thanks for the suggestion. I do notice that.But the result is the same
> though.
>
>
>> 2. Job history server only shows the finished mapreduce jobs. There will
>> be another application history server that shows all completed applications
>> run by YARN, but it's not available on 2.2.
>>
>> 3. ResourceManager webUI is not the job history web UI. You should check
>> your yarn-site.xml to see what's the address of the RM webUI. It will list
>> all the applications that RM remembers.
>>
>
>
>
>> - Zhijie
>>
>>
>> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>>
>>> Dear group,
>>>
>>>
>>>
>>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>>> job -list or hadoop job -list all, it throws a NPE like this:
>>>
>>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>>> deprecated. Instead, use dfs.metrics.session-id
>>>
>>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>>> processName=JobTracker, sessionId=
>>>
>>> Exception in thread "main" java.lang.NullPointerException
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>>
>>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>
>>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>>
>>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>>> It shows no job was running and no job finishing although I was running
>>> jobs.
>>>
>>> Please help me to solve this problem.
>>>
>>> Thanks!!
>>>
>>>
>>>
>>> Richard Chen
>>>
>>>
>>>
>>
>>
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
> Dislike trivia, appreciate no-nonsense!
>
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
RC: RM website address inside yarn-site.xml was corrected put there.
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
RC: RM website address inside yarn-site.xml was corrected put there.
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Richard Chen <cx...@gmail.com>.
Thanks for the reply. Due to company mail filtering issue, I decided to use
my personal mail account so that I can attach files freely in the future.
Please see my reply inline to your questions below.
Richard
On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Richard,
>
> Not sure how NPE happened on you command line, but I'd like to clarify
> something here:
> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop
> job" is deprecated. If you want to see all kinds of applications run by
> your YARN cluster, please use "yarn application".
>
> Thanks for the suggestion. I do notice that.But the result is the same
though.
> 2. Job history server only shows the finished mapreduce jobs. There will
> be another application history server that shows all completed applications
> run by YARN, but it's not available on 2.2.
>
> 3. ResourceManager webUI is not the job history web UI. You should check
> your yarn-site.xml to see what's the address of the RM webUI. It will list
> all the applications that RM remembers.
>
RC: RM website address inside yarn-site.xml was corrected put there.
> - Zhijie
>
>
> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
>
>> Dear group,
>>
>>
>>
>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
>> job -list or hadoop job -list all, it throws a NPE like this:
>>
>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
>> deprecated. Instead, use dfs.metrics.session-id
>>
>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>>
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>>
>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>
>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>>
>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
>> It shows no job was running and no job finishing although I was running
>> jobs.
>>
>> Please help me to solve this problem.
>>
>> Thanks!!
>>
>>
>>
>> Richard Chen
>>
>>
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Zhijie Shen <zs...@hortonworks.com>.
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify
something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job"
is deprecated. If you want to see all kinds of applications run by your
YARN cluster, please use "yarn application".
2. Job history server only shows the finished mapreduce jobs. There will be
another application history server that shows all completed applications
run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check
your yarn-site.xml to see what's the address of the RM webUI. It will list
all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
> Dear group,
>
>
>
> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
> job -list or hadoop job -list all, it throws a NPE like this:
>
> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
> deprecated. Instead, use dfs.metrics.session-id
>
> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
>
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>
> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
> It shows no job was running and no job finishing although I was running
> jobs.
>
> Please help me to solve this problem.
>
> Thanks!!
>
>
>
> Richard Chen
>
>
>
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Zhijie Shen <zs...@hortonworks.com>.
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify
something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job"
is deprecated. If you want to see all kinds of applications run by your
YARN cluster, please use "yarn application".
2. Job history server only shows the finished mapreduce jobs. There will be
another application history server that shows all completed applications
run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check
your yarn-site.xml to see what's the address of the RM webUI. It will list
all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
> Dear group,
>
>
>
> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
> job -list or hadoop job -list all, it throws a NPE like this:
>
> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
> deprecated. Instead, use dfs.metrics.session-id
>
> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
>
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>
> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
> It shows no job was running and no job finishing although I was running
> jobs.
>
> Please help me to solve this problem.
>
> Thanks!!
>
>
>
> Richard Chen
>
>
>
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Zhijie Shen <zs...@hortonworks.com>.
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify
something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job"
is deprecated. If you want to see all kinds of applications run by your
YARN cluster, please use "yarn application".
2. Job history server only shows the finished mapreduce jobs. There will be
another application history server that shows all completed applications
run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check
your yarn-site.xml to see what's the address of the RM webUI. It will list
all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
> Dear group,
>
>
>
> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
> job -list or hadoop job -list all, it throws a NPE like this:
>
> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
> deprecated. Instead, use dfs.metrics.session-id
>
> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
>
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>
> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
> It shows no job was running and no job finishing although I was running
> jobs.
>
> Please help me to solve this problem.
>
> Thanks!!
>
>
>
> Richard Chen
>
>
>
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: No job shown in Hadoop resource manager web UI when running jobs
in the cluster
Posted by Zhijie Shen <zs...@hortonworks.com>.
Hi Richard,
Not sure how NPE happened on you command line, but I'd like to clarify
something here:
1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job"
is deprecated. If you want to see all kinds of applications run by your
YARN cluster, please use "yarn application".
2. Job history server only shows the finished mapreduce jobs. There will be
another application history server that shows all completed applications
run by YARN, but it's not available on 2.2.
3. ResourceManager webUI is not the job history web UI. You should check
your yarn-site.xml to see what's the address of the RM webUI. It will list
all the applications that RM remembers.
- Zhijie
On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard <ri...@intel.com>wrote:
> Dear group,
>
>
>
> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop
> job -list or hadoop job -list all, it throws a NPE like this:
>
> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is
> deprecated. Instead, use dfs.metrics.session-id
>
> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
>
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504)
>
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
>
> and on hadoop webapp like jobhistory ( I turn on the jobhistory server).
> It shows no job was running and no job finishing although I was running
> jobs.
>
> Please help me to solve this problem.
>
> Thanks!!
>
>
>
> Richard Chen
>
>
>
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.