[jira] Updated: (MAPREDUCE-959) JobConf::setWorkingDirectory requires that the default FileSystem is reachable

["Chris Douglas (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated MAPREDUCE-959:
------------------------------------

    Attachment: M959-test.patch

Unit test illustrating the issue.

Arguably, this is an issue with new instances of DistributedFileSystem and protocol compatibility should be established lazily rather than in the constructor.

> JobConf::setWorkingDirectory requires that the default FileSystem is reachable
> ------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-959
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-959
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, test
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: M959-test.patch
>
>
> If mapred.working.dir is not set, JobConf::setWorkingDirectory will attempt to obtain the default working directory for the default FileSystem. In trunk at least, if the default fs is HDFS and not reachable, set will fail:
> {noformat}
> java.net.UnknownHostException: unknown host: notahost
> java.lang.RuntimeException: java.net.UnknownHostException: unknown host: notahost
>   at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:541)
>   at org.apache.hadoop.mapred.JobConf.setWorkingDirectory(JobConf.java:522)
>   at org.apache.hadoop.conf.TestJobConf.testSetWorkingDir(TestJobConf.java:67)
> Caused by: java.net.UnknownHostException: unknown host: notahost
>   at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:216)
>   at org.apache.hadoop.ipc.Client.getConnection(Client.java:876)
>   at org.apache.hadoop.ipc.Client.call(Client.java:746)
>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:223)
>   at $Proxy4.getProtocolVersion(Unknown Source)
>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:366)
>   at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:169)
>   at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:276)
>   at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:235)
>   at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:83)
>   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1430)
>   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:69)
>   at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1458)
>   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1446)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:190)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:98)
>   at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:537)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.