You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Chris Nauroth (JIRA)" <ji...@apache.org> on 2015/08/05 21:51:05 UTC

[jira] [Updated] (HADOOP-12304) Applications using FileContext fail with the default file system configured to be wasb/s3/etc.

     [ https://issues.apache.org/jira/browse/HADOOP-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Nauroth updated HADOOP-12304:
-----------------------------------
    Attachment: HADOOP-12304.001.patch

The attached patch changes {{DelegateToFileSystem}} to translate the 0 returned from {{FileSystem#getDefaultPort}} to the -1 expected by {{AbstractFileSystem#getUri}}.  I verified that the tests written for HADOOP-11618 still pass.  I also added a new test to confirm that using wasb as the default file system uses a correct URL (without port :0) when accessed through {{FileContext}}.  I also have done manual testing to confirm that I can run MapReduce jobs with wasb as the default file system after this change.

> Applications using FileContext fail with the default file system configured to be wasb/s3/etc.
> ----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-12304
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12304
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.7.0, 2.7.1
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>            Priority: Blocker
>              Labels: 2.7.2-candidate
>         Attachments: HADOOP-12304.001.patch
>
>
> HADOOP-11618 fixed a bug with {{DelegateToFileSystem}} using the wrong default port.  As a side effect of this patch, file path URLs that previously had no port now insert :0 for the port, as per the default implementation of {{FileSystem#getDefaultPort}}.  At runtime, this can cause an application to erroneously try contacting port 0 for a remote blob store service.  The connection fails.  Ultimately, this renders wasb, s3, and probably custom file system implementations outside the Hadoop source tree completely unusable as the default file system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)