You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ivan Mitic (JIRA)" <ji...@apache.org> on 2012/10/02 05:41:07 UTC
[jira] [Created] (HADOOP-8868) FileUtil#symlink and FileUtil#chmod
should normalize the path before calling into shell APIs
Ivan Mitic created HADOOP-8868:
----------------------------------
Summary: FileUtil#symlink and FileUtil#chmod should normalize the path before calling into shell APIs
Key: HADOOP-8868
URL: https://issues.apache.org/jira/browse/HADOOP-8868
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 1-win
Reporter: Ivan Mitic
Assignee: Ivan Mitic
We have seen cases where paths passed in from FileUtil#symlink or FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8868:
-------------------------------
Status: Patch Available (was: Open)
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8868:
-------------------------------
Attachment: HADOOP-8868.branch-1-win.chmod.patch
Attaching the patch.
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8868) FileUtil#chmod should normalize
the path before calling into shell APIs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472140#comment-13472140 ]
Hadoop QA commented on HADOOP-8868:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548362/HADOOP-8868.branch-1-win.chmod.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1586//console
This message is automatically generated.
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8868:
------------------------------------
Resolution: Fixed
Fix Version/s: 1-win
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1 for the change. I committed the patch to branch-1-win. Thank you Ivan.
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Fix For: 1-win
>
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8868) FileUtil#chmod should normalize
the path before calling into shell APIs
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473914#comment-13473914 ]
Ivan Mitic commented on HADOOP-8868:
------------------------------------
Thanks Bikas for the review.
bq. So we are using JAVA API to resolve the path to a normalized form? Ideally the FileUtil method could take File arguments instead of strings but we'd like to avoid changing the public API.
Right. Having File APIs would be great (as Java polishes this up nicely), however, for compat reasons this makes more sense.
bq. In what cases can we get a mix of slashes on the string path?
In one case, a path with only forward slashes was sent to winutils. The path was longer to MAX_PATH (260 chars) so it was prepended with "\\?\" to tell the OS that the path is longer than 260. In this scenario we had both forward and backward slashes. Now, it makes more sense to do the slash conversion in Java then in winutils, given that Java provides better/tested cross-platform support. Make sense?
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8868:
-------------------------------
Description:
We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
This causes problems, since some Windows APIs do not work well with mixed slashes.
was:
We have seen cases where paths passed in from FileUtil#symlink or FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
This causes problems, since some Windows APIs do not work well with mixed slashes.
Summary: FileUtil#chmod should normalize the path before calling into shell APIs (was: FileUtil#symlink and FileUtil#chmod should normalize the path before calling into shell APIs)
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8868:
-------------------------------
Status: Patch Available (was: Open)
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8868) FileUtil#chmod should normalize
the path before calling into shell APIs
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472952#comment-13472952 ]
Bikas Saha commented on HADOOP-8868:
------------------------------------
So we are using JAVA API to resolve the path to a normalized form? Ideally the FileUtil method could take File arguments instead of strings but we'd like to avoid changing the public API.
In what cases can we get a mix of slashes on the string path?
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8868) FileUtil#chmod should normalize
the path before calling into shell APIs
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475279#comment-13475279 ]
Bikas Saha commented on HADOOP-8868:
------------------------------------
Ok. This also avoids cases where a path is composed of a root path from config/default which might contain / and a subpath from locaFS that contains a \.
+1
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8868) FileUtil#chmod should normalize the
path before calling into shell APIs
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8868:
------------------------------------
Status: Open (was: Patch Available)
> FileUtil#chmod should normalize the path before calling into shell APIs
> -----------------------------------------------------------------------
>
> Key: HADOOP-8868
> URL: https://issues.apache.org/jira/browse/HADOOP-8868
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 1-win
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8868.branch-1-win.chmod.patch
>
>
> We have seen cases where paths passed in from FileUtil#chmod to Shell APIs can contain both forward and backward slashes on Windows.
> This causes problems, since some Windows APIs do not work well with mixed slashes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira