[jira] [Created] (HDFS-8353) FileSystem.resolvePath() fails for Windows UNC paths

["john lilley (JIRA)" <ji...@apache.org>]

john lilley created HDFS-8353:
---------------------------------

             Summary: FileSystem.resolvePath() fails for Windows UNC paths
                 Key: HDFS-8353
                 URL: https://issues.apache.org/jira/browse/HDFS-8353
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: HDFS
    Affects Versions: 2.6.0, 2.4.0
         Environment: Windows 8, x64
Java 1.7
            Reporter: john lilley


FileSystem.resolvePath() fails with Windows UNC path. This has a knock-on effect with Parquet file access in local filesystem, because Parquet has no open-using-stream API that we've been able to find. Take this simple test: 

public class Scratch {
  public static void main(String[] args) {
    // Note that this path must exist
    java.net.URI uriWithAuth = java.net.URI.create("file://host/share/file");
    try {
      FileSystem fs = FileSystem.get(uriWithAuth, new Configuration());
      fs.resolvePath(new Path(uriWithAuth));
    } catch (Exception ex) {
      ex.printStackTrace();
    }
  }
}

The resolvePath() call will fail in FileSystem.checkPath():
  protected void checkPath(Path path) {
    URI uri = path.toUri();
    String thatScheme = uri.getScheme();
    if (thatScheme == null)                // fs is relative
      return;
    URI thisUri = getCanonicalUri();
    String thisScheme = thisUri.getScheme();
    //authority and scheme are not case sensitive
    if (thisScheme.equalsIgnoreCase(thatScheme)) {// schemes match
      String thisAuthority = thisUri.getAuthority();
      String thatAuthority = uri.getAuthority();
      if (thatAuthority == null &&                // path's authority is null
          thisAuthority != null) {                // fs has an authority
        URI defaultUri = getDefaultUri(getConf());
        if (thisScheme.equalsIgnoreCase(defaultUri.getScheme())) {
          uri = defaultUri; // schemes match, so use this uri instead
        } else {
          uri = null; // can't determine auth of the path
        }
      }
      if (uri != null) {
        // canonicalize uri before comparing with this fs
        uri = canonicalizeUri(uri);
        thatAuthority = uri.getAuthority();
        if (thisAuthority == thatAuthority ||       // authorities match
            (thisAuthority != null &&
             thisAuthority.equalsIgnoreCase(thatAuthority)))
          return;
      }
    }
    throw new IllegalArgumentException("Wrong FS: "+path+
                                       ", expected: "+this.getUri());
  }


The problem is that thisAuthority is null, and thatAuthority gets "host". There is no logic for dealing with that case. In fact, this method seems broken in several ways. There are at least two problems: 
-- For UNC paths like file://host/share/... , the authority does not need to match, at least for Windows UNC paths. All of these paths are the same file system: "file:///F:/folder/file", "file://host1/share/file", "file://host2/share/file". 
-- The test thisAuthority == thatAuthority violates Java 101. It should be thisAuthority.equals(thatAuthority) 
-- hostnames are case-independent, so I think that the authority comparison should also be case-insensitive, at least for UNC paths. 
-- I don't see any attempt to resolve hostnames to IP addresses, but that may simply be beyond the scope of this method.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)