[jira] [Created] (TEZ-3993) Tez fails to parse windows file paths in local mode

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

Jonathan Eagles created TEZ-3993:
------------------------------------

             Summary: Tez fails to parse windows file paths in local mode
                 Key: TEZ-3993
                 URL: https://issues.apache.org/jira/browse/TEZ-3993
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Jonathan Eagles
            Assignee: Jonathan Eagles


TezLocalCacheManager tries to generate symlinks to files that it puts in the local cache, but the code that it uses to construct the path names is not safe on Windows and causes bad file names to be constructed when run in a Windows environment. On Windows, a path like file:/c:/path/to/my/file should be legal and transform to c:\path\to\my\file, but with the invalid construct, it turns into the illegal value /c:/path/to/my/file instead.

In TezLocalCacheManager, there is code that does

{code}
private boolean createSymlink(Path target, Path link) throws IOException {
    LOG.info("Creating symlink: {} <- {}", target, link);
    String targetPath = target.toUri().getPath();
    String linkPath = link.toUri().getPath();
{code}

It looks like there are several other places in the Tez code that also use the Path.toUri().getPath() construct that probably also need to be fixed in order to work correctly on Windows.

The construct Path.toUri().getPath() doesn't handle windows directories correctly. The Java File class understands how to do this correctly, so this should really be replaced by

{code}
private boolean createSymlink(Path target, Path link) throws IOException {
    LOG.info("Creating symlink: {} <- {}", target, link);
    String targetPath = new File(target.toUri()).getCanonicalPath();
    String linkPath = new File(link.toUri()).getCanonicalPath();
{code}
{code}
2018-09-19T16:32:53,287 ERROR [LocalContainerLauncher-SubTaskRunner] org.apache.tez.dag.app.launcher.LocalContainerLauncher - TezSubTaskRunner failed due to exception
java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/...fullpath
        at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
        at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
        at java.nio.file.Paths.get(Paths.java:84)
        at org.apache.tez.dag.app.launcher.TezLocalCacheManager.createSymlink(TezLocalCacheManager.java:173)
        at org.apache.tez.dag.app.launcher.TezLocalCacheManager.localize(TezLocalCacheManager.java:126)
        at org.apache.tez.dag.app.launcher.LocalContainerLauncher.launch(LocalContainerLauncher.java:263)
        at org.apache.tez.dag.app.launcher.LocalContainerLauncher.access$300(LocalContainerLauncher.java:82)
        at org.apache.tez.dag.app.launcher.LocalContainerLauncher$TezSubTaskRunner.run(LocalContainerLauncher.java:207)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)