You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Frank Luo <jl...@merkleinc.com> on 2013/05/28 18:49:01 UTC
how does hive find where is MR job tracker
I have a cloudera cluster, version 4.2.0.
In the hive configuration, I have "MapReduce Service" set to "mapreduce1", which is my MR service.
However, without setting "mapred.job.tracker", whenever I run hive command, it always sends the job to a wrong job tracker. Here is the error:
java.net.ConnectException: Call From hqhd01ed01.pclc0.merkle.local/10.129.2.52 to hqhd01ed01.pclc0.merkle.local:8021 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
And the Cloudera Manager doesn't allow me to manually set "mapred.job.tracker". So my question is how to make Hive point to the right job tracker without setting ""mapred.job.tracker" every time.
PS. Not sure it matters, but I did move the job tracker from machine A to machine B.
Thx!
RE: how does hive find where is MR job tracker
Posted by Frank Luo <jl...@merkleinc.com>.
Thanks for reply.
Yes, I had the old server name in mapred-site.xml. Odd enough, I couldn't find a way to update the file through CM.
From: Sanjay Subramanian [mailto:Sanjay.Subramanian@wizecommerce.com]
Sent: Tuesday, May 28, 2013 12:08 PM
To: user@hive.apache.org; bejoy_ks@yahoo.com
Subject: Re: how does hive find where is MR job tracker
In Cloudera Manager , there is a Safety Valve feature (its a multiline text widget) that u can use to input the XML properties that u would use for mapred-site.xml
Possibly since u changed the JobTracker machine , u have to mod the mapred-site.xml to specify the machine name and port
Regards
Sanjay
From: "bejoy_ks@yahoo.com<ma...@yahoo.com>" <be...@yahoo.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "bejoy_ks@yahoo.com<ma...@yahoo.com>" <be...@yahoo.com>>
Date: Tuesday, May 28, 2013 10:02 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: how does hive find where is MR job tracker
Hive gets the JobTracker from the mapred-site.xml specified within your $HADOOP_HOME/conf.
Is your $HADOOP_HOME/conf/mapred-site.xml on the node that runs hive have the correct value for jobtracker?
If not changing that to the right one might resolve your issue.
Regards
Bejoy KS
Sent from remote device, Please excuse typos
________________________________
From: Frank Luo <jl...@merkleinc.com>>
Date: Tue, 28 May 2013 16:49:01 +0000
To: user@hive.apache.org<ma...@hive.apache.org>>
ReplyTo: user@hive.apache.org<ma...@hive.apache.org>
Subject: how does hive find where is MR job tracker
I have a cloudera cluster, version 4.2.0.
In the hive configuration, I have "MapReduce Service" set to "mapreduce1", which is my MR service.
However, without setting "mapred.job.tracker", whenever I run hive command, it always sends the job to a wrong job tracker. Here is the error:
java.net.ConnectException: Call From hqhd01ed01.pclc0.merkle.local/10.129.2.52 to hqhd01ed01.pclc0.merkle.local:8021 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
And the Cloudera Manager doesn't allow me to manually set "mapred.job.tracker". So my question is how to make Hive point to the right job tracker without setting ""mapred.job.tracker" every time.
PS. Not sure it matters, but I did move the job tracker from machine A to machine B.
Thx!
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: how does hive find where is MR job tracker
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
In Cloudera Manager , there is a Safety Valve feature (its a multiline text widget) that u can use to input the XML properties that u would use for mapred-site.xml
Possibly since u changed the JobTracker machine , u have to mod the mapred-site.xml to specify the machine name and port
Regards
Sanjay
From: "bejoy_ks@yahoo.com<ma...@yahoo.com>" <be...@yahoo.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "bejoy_ks@yahoo.com<ma...@yahoo.com>" <be...@yahoo.com>>
Date: Tuesday, May 28, 2013 10:02 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: how does hive find where is MR job tracker
Hive gets the JobTracker from the mapred-site.xml specified within your $HADOOP_HOME/conf.
Is your $HADOOP_HOME/conf/mapred-site.xml on the node that runs hive have the correct value for jobtracker?
If not changing that to the right one might resolve your issue.
Regards
Bejoy KS
Sent from remote device, Please excuse typos
________________________________
From: Frank Luo <jl...@merkleinc.com>>
Date: Tue, 28 May 2013 16:49:01 +0000
To: user@hive.apache.org<ma...@hive.apache.org>>
ReplyTo: user@hive.apache.org<ma...@hive.apache.org>
Subject: how does hive find where is MR job tracker
I have a cloudera cluster, version 4.2.0.
In the hive configuration, I have “MapReduce Service” set to “mapreduce1”, which is my MR service.
However, without setting “mapred.job.tracker”, whenever I run hive command, it always sends the job to a wrong job tracker. Here is the error:
java.net.ConnectException: Call From hqhd01ed01.pclc0.merkle.local/10.129.2.52 to hqhd01ed01.pclc0.merkle.local:8021 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
And the Cloudera Manager doesn’t allow me to manually set “mapred.job.tracker”. So my question is how to make Hive point to the right job tracker without setting ““mapred.job.tracker” every time.
PS. Not sure it matters, but I did move the job tracker from machine A to machine B.
Thx!
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: how does hive find where is MR job tracker
Posted by be...@yahoo.com.
Hive gets the JobTracker from the mapred-site.xml specified within your $HADOOP_HOME/conf.
Is your $HADOOP_HOME/conf/mapred-site.xml on the node that runs hive have the correct value for jobtracker?
If not changing that to the right one might resolve your issue.
Regards
Bejoy KS
Sent from remote device, Please excuse typos
-----Original Message-----
From: Frank Luo <jl...@merkleinc.com>
Date: Tue, 28 May 2013 16:49:01
To: user@hive.apache.org<us...@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: how does hive find where is MR job tracker
I have a cloudera cluster, version 4.2.0.
In the hive configuration, I have "MapReduce Service" set to "mapreduce1", which is my MR service.
However, without setting "mapred.job.tracker", whenever I run hive command, it always sends the job to a wrong job tracker. Here is the error:
java.net.ConnectException: Call From hqhd01ed01.pclc0.merkle.local/10.129.2.52 to hqhd01ed01.pclc0.merkle.local:8021 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
And the Cloudera Manager doesn't allow me to manually set "mapred.job.tracker". So my question is how to make Hive point to the right job tracker without setting ""mapred.job.tracker" every time.
PS. Not sure it matters, but I did move the job tracker from machine A to machine B.
Thx!