[jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks

["Chris Nauroth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630513#comment-13630513 ] 

Chris Nauroth commented on HADOOP-9043:
---------------------------------------

I was just trying to resume progress on this to fix the remaining test failures related to Windows symlinks on the local file system.  I discovered a new problem.  The trunk code still has a Unix command dependency on the readlink command for determining the target of a symlink.  (See {{RawLocalFs#readLink}}.)  This is the source of some of the test failures.

I propose the addition of a new "winutils readlink" command.  The logic of this command would be:

# Call {{CreateFile}} with {{OPEN_EXISTING}} and {{FILE_FLAG_OPEN_REPARSE_POINT}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx
# Call {{GetFileInformationByHandle}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa364952(v=vs.85).aspx
# Check the {{BY_HANDLE_FILE_INFORMATION}} for the presence of {{FILE_ATTRIBUTE_REPARSE_POINT}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa363788(v=vs.85).aspx
** http://msdn.microsoft.com/en-us/library/windows/desktop/gg258117(v=vs.85).aspx
# If not a reparse point, exit with code 1 and print nothing to stdout.  (This is what Unix readlink does.)
# Call {{DeviceIoControl}} with {{FSCTL_GET_REPARSE_POINT}}.
** http://msdn.microsoft.com/en-us/library/aa363216(v=VS.85).aspx
** http://msdn.microsoft.com/en-us/library/aa364571.aspx
# Get the {{REPARSE_DATA_BUFFER}} structure.
** http://msdn.microsoft.com/en-us/library/ff552012.aspx
# Check if {{ReparseTag}} is {{IO_REPARSE_TAG_SYMLINK}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa365511(v=vs.85).aspx
# If not {{IO_REPARSE_TAG_SYMLINK}}, then...?
# Get target from {{SymbolicLinkReparseBuffer}}.
# Print target to stdout and exit with code 0.  (This is what Unix readlink does.)

Could someone with more Windows expertise review this and comment on whether or not the logic looks correct?  There are a few edge cases that I'm not sure how to handle.  What should we do if the reparse point is not a symlink (i.e. junction point)?  MSDN also mentions that some reparse points may have a different data structure associated with them, a {{REPARSE_GUID_DATA_BUFFER}}, and I'm not sure what special handling is required around that.

                
> winutils can create unusable symlinks
> -------------------------------------
>
>                 Key: HADOOP-9043
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9043
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 3.0.0, 1-win
>            Reporter: Chris Nauroth
>            Assignee: Arpit Agarwal
>             Fix For: 3.0.0, 1-win
>
>         Attachments: HADOOP-9043.branch-1.2.patch, HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, HADOOP-9043.trunk.patch
>
>
> In general, the winutils symlink command rejects attempts to create symlinks targeting a destination file that does not exist.  However, if given a symlink destination with forward slashes pointing at a file that does exist, then it creates the symlink with the forward slashes, and then attempts to open the file through the symlink will fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira