You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2015/09/01 21:17:45 UTC
[jira] [Commented] (MAPREDUCE-6463) AM should register RM using IP
address instead of hostname
[ https://issues.apache.org/jira/browse/MAPREDUCE-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725910#comment-14725910 ]
Steve Loughran commented on MAPREDUCE-6463:
-------------------------------------------
Even though its now been resolved as a WONTFIX, I'll repeat a key reason: Kerberos. Most new clusters are coming up kerberized, and if people start using or relying on IP addresses things just won't work there. It's bad enough we have too many codepaths for {{isSecure()}} to have different network settings.
There are still places where Hadoop does use IPAddrs where it should be using names; this will surface in the big IPv6 support branch (which will be optional)
> AM should register RM using IP address instead of hostname
> ----------------------------------------------------------
>
> Key: MAPREDUCE-6463
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6463
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jun Gong
> Assignee: Jun Gong
> Attachments: MAPREDUCE-6463.001.patch
>
>
> I copied hadoop direcotry to a host which is not in cluster, and ran a streaming job on it. I encountered following error:
> {noformat}
> 15/08/25 18:48:07 INFO impl.YarnClientImpl: Submitted application application_1440039112410_1578
> 15/08/25 18:48:07 INFO mapreduce.JobSubmitter: JobID:job_1440039112410_1578 ClientSubmitJob time:692
> 15/08/25 18:48:07 INFO mapreduce.Job: The url to track the job: http://10.222.7.210:8080/proxy/application_1440039112410_1578/
> 15/08/25 18:48:07 INFO mapreduce.Job: Running job: job_1440039112410_1578
> 15/08/25 18:48:13 INFO mapred.ClientServiceDelegate: Connecting to szsk-ad-serving-10-222-7-204:41486
> 15/08/25 18:48:13 INFO mapred.ClientServiceDelegate: Failed to contact AM/History for job job_1440039112410_1578 retrying..
> java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "szsk-ad-serving-10-222-7-204":41486; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:742)
> at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:400)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1452)
> at org.apache.hadoop.ipc.Client.call(Client.java:1381)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy16.getJobReport(Unknown Source)
> at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:350)
> at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:517)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:536)
> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:318)
> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:315)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:315)
> at org.apache.hadoop.mapreduce.Job.isComplete(Job.java:604)
> at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1337)
> at org.apache.hadoop.mapred.JobClient$NetworkedJob.monitorAndPrintJob(JobClient.java:409)
> at org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:858)
> at org.apache.hadoop.streaming.StreamJob.submitAndMonitorJob(StreamJob.java:1018)
> at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:135)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: java.net.UnknownHostException
> ... 34 more
> {noformat}
> The exception is because AM register RM with hostname 'szsk-ad-serving-10-222-7-204' and client could not resolve it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)