[jira] [Issue Comment Deleted] (TEZ-3993) Tez fails to parse windows file paths in local mode

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Eagles updated TEZ-3993:
---------------------------------
    Comment: was deleted

(was: | (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} maven {color} | {color:red}  0m  3s{color} | {color:red} ERROR: maven was not available. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | TEZ-3993 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12941240/TEZ-3993.002.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  checkstyle  compile  |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15 17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/yetus/precommit/personality/tez.sh |
| git revision | master / 64c04f1 |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build-Yetus/3/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.

)

> Tez fails to parse windows file paths in local mode
> ---------------------------------------------------
>
>                 Key: TEZ-3993
>                 URL: https://issues.apache.org/jira/browse/TEZ-3993
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>            Priority: Major
>         Attachments: TEZ-3993.001.patch, TEZ-3993.002.patch
>
>
> TezLocalCacheManager tries to generate symlinks to files that it puts in the local cache, but the code that it uses to construct the path names is not safe on Windows and causes bad file names to be constructed when run in a Windows environment. On Windows, a path like file:/c:/path/to/my/file should be legal and transform to c:\path\to\my\file, but with the invalid construct, it turns into the illegal value /c:/path/to/my/file instead.
> In TezLocalCacheManager, there is code that does
> {code}
> private boolean createSymlink(Path target, Path link) throws IOException {
>     LOG.info("Creating symlink: {} <- {}", target, link);
>     String targetPath = target.toUri().getPath();
>     String linkPath = link.toUri().getPath();
> {code}
> It looks like there are several other places in the Tez code that also use the Path.toUri().getPath() construct that probably also need to be fixed in order to work correctly on Windows.
> The construct Path.toUri().getPath() doesn't handle windows directories correctly. The Java File class understands how to do this correctly, so this should really be replaced by
> {code}
> private boolean createSymlink(Path target, Path link) throws IOException {
>     LOG.info("Creating symlink: {} <- {}", target, link);
>     String targetPath = new File(target.toUri()).getCanonicalPath();
>     String linkPath = new File(link.toUri()).getCanonicalPath();
> {code}
> {code}
> 2018-09-19T16:32:53,287 ERROR [LocalContainerLauncher-SubTaskRunner] org.apache.tez.dag.app.launcher.LocalContainerLauncher - TezSubTaskRunner failed due to exception
> java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/...fullpath
>         at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
>         at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
>         at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
>         at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
>         at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
>         at java.nio.file.Paths.get(Paths.java:84)
>         at org.apache.tez.dag.app.launcher.TezLocalCacheManager.createSymlink(TezLocalCacheManager.java:173)
>         at org.apache.tez.dag.app.launcher.TezLocalCacheManager.localize(TezLocalCacheManager.java:126)
>         at org.apache.tez.dag.app.launcher.LocalContainerLauncher.launch(LocalContainerLauncher.java:263)
>         at org.apache.tez.dag.app.launcher.LocalContainerLauncher.access$300(LocalContainerLauncher.java:82)
>         at org.apache.tez.dag.app.launcher.LocalContainerLauncher$TezSubTaskRunner.run(LocalContainerLauncher.java:207)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)