You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Bill Habermaas (JIRA)" <ji...@apache.org> on 2009/07/20 23:02:15 UTC
[jira] Commented: (MAPREDUCE-438) When connecting to HDFS using an
IP Task MapRed gets confused when checking the output path.
[ https://issues.apache.org/jira/browse/MAPREDUCE-438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733353#action_12733353 ]
Bill Habermaas commented on MAPREDUCE-438:
------------------------------------------
This is likely still an issue. I've fixed it on my 0.18.3 but not tried on anything more recent as I am locked into java 1.5. Is anybody else besides me interested in fixing this?
> When connecting to HDFS using an IP Task MapRed gets confused when checking the output path.
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-438
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-438
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: AIX 6.1 and Solaris
> Reporter: Bill Habermaas
> Priority: Minor
>
> The exception appearing below happens if you are using an IP address to reference a HDFS file system path. Although the URI authority for either path (whether referenced by hostname:port or IP-address:port) refer to the same filesystem, Task throws an exception because the stringified comparisons of the authority in the URIs are not equal. The FileSystem holds the network location of the namenode as a host name regardless as to whether an IP address or hostname is used when the file system is connected (refer to HADOOP-5191 for an earlier fix specifically for HDFS only).
> 2009-03-27 04:15:45,045 WARN [Thread-145] org.apache.hadoop.mapred.LocalJobRunner: job_local_0002
> java.io.IOException: Can not get the relative path: base = hdfs://10.120.16.68:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0 child = hdfs://p520aix61.mydomain.com:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0/part-00000
> at org.apache.hadoop.mapred.Task.getFinalPath(Task.java:586)
> at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:599)
> at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:617)
> at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:561)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:202)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.