You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Chuan Liu (JIRA)" <ji...@apache.org> on 2012/05/31 18:51:23 UTC
[jira] [Created] (HADOOP-8456) Support spaces in user names and
group names.
Chuan Liu created HADOOP-8456:
---------------------------------
Summary: Support spaces in user names and group names.
Key: HADOOP-8456
URL: https://issues.apache.org/jira/browse/HADOOP-8456
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 1.1.0, 0.24.0
Reporter: Chuan Liu
Priority: Minor
When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8456:
------------------------------------
Resolution: Fixed
Fix Version/s: 1-win
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1. I committed the patch to branch-1-win. Thank you Ivan.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Fix For: 1-win
>
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498836#comment-13498836 ]
Daryn Sharp commented on HADOOP-8456:
-------------------------------------
+1 If others are ok with it.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8456) Support spaces in user names and
group names.
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic reassigned HADOOP-8456:
----------------------------------
Assignee: Ivan Mitic
> Support spaces in user names and group names.
> ---------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1.1.0, 0.24.0
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8456:
-------------------------------
Summary: Support spaces in user names and group names in results returned via winutils (was: Support spaces in user names and group names.)
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names.
Posted by "Chuan Liu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chuan Liu updated HADOOP-8456:
------------------------------
Component/s: native
> Support spaces in user names and group names.
> ---------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1.1.0, 0.24.0
> Reporter: Chuan Liu
> Priority: Minor
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484769#comment-13484769 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
Thanks all for the feedback! My answers are below.
bq. AccessControlList depends on a space delimiter, you will need to somehow address that too.
You bring up a good poing Vinod. I discussed this part with the team before posting a patch. At this point, we will keep the current HDFS user and group name semantics, so no spaces. If we decide to make this change, that would be a separate Jira. The goal of this patch is to provide accurate information to Hadoop in the first place since providing false information can lead some other problems. Does this make sense?
bq. 1.Does the wrapping of double quotes around the command args imply that the windows shell is re-parsing the group name? If so, that would seem to be a security issue if the user is somehow able to inject input with metachars.
Daryn, I am relatively confident that Windows deals with this stuff properly. I would assume the same for the thin JNI layer. Passing in paramaters under quotes means that they are passed as separate args to app underneath (C argc/argv). Do you maybe know of some problems here?
bq. 2.Maybe turn the delimiter into a general option so it's not hardcoded throughout the patch?
Make sense, I'll prepare the updated patch.
bq. How does Cygwin, MS SUA, etc handle this problem? We should do the same thing on Windows in order maintain compatibility with those systems.
Thanks Allen. I just checked using Cygwin's {{id -Gn}}, and it seems that Cygwin keeps some internal metadata about groups. I know this was also true for permissions/ownership.
{code}
>id -Gn ivanmi
mkpasswd
{code}
Actual groups:
{code}
>bin\winutils.exe groups ivanmi
AutoSql Admin Group HelpLibraryUpdaters Administrators
{code}
And now with the separator:
{code}
>bin\winutils.exe groups -F ivanmi
AutoSql Admin Group|HelpLibraryUpdaters|Administrators
{code}
Generally, we want to natively integrate Hadoop with Windows, in a sense that Windows user/group information flows thru Hadoop.
Let me know if this makes sense.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483810#comment-13483810 ]
Hadoop QA commented on HADOOP-8456:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12550730/HADOOP-8456.branch-1-win.spaces.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1672//console
This message is automatically generated.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485242#comment-13485242 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
bq. Quotes aren't adequate if the shell is involved. Ex. A typical exploit is to pass constructs like ";malicious-cmd;" or $(malicious-cmd). If the shell is completely bypassed, as it should be, the list of args can be passed through execve with no quotes. The shell should never be involved in command execution unless you actually want it to possibly mangle your parameters which is almost never the case.
I see, thanks for clarifying. I don't think we have this problem here as windows shell is not involved. The quoted strings go thru the following stack: Java#ProcessBuilder->Win32#CreateProcess->Winutils.exe#argc/argv. On the other hand, we have to place username under quotes, as it won't be passed as a single argument to underlying winutils exe. For example:
{{winutils groups user name}}
does not work (4 args), while
{{winutils groups "user name"}}
works. Make sense, or am I missing something?
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484950#comment-13484950 ]
Daryn Sharp commented on HADOOP-8456:
-------------------------------------
bq. Passing in paramaters under quotes means that they are passed as separate args to app underneath (C argc/argv). Do you maybe know of some problems here?
Quotes aren't adequate if the shell is involved. Ex. A typical exploit is to pass constructs like {{";malicious-cmd;"}} or {{$(malicious-cmd)}}. If the shell is completely bypassed, as it should be, the list of args can be passed through execve with no quotes. The shell should never be involved in command execution unless you actually want it to possibly mangle your parameters which is almost never the case.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486053#comment-13486053 ]
Daryn Sharp commented on HADOOP-8456:
-------------------------------------
Let me investigate a bit further.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488695#comment-13488695 ]
Hadoop QA commented on HADOOP-8456:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12551677/HADOOP-8456.branch-1-win.spaces.2.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1694//console
This message is automatically generated.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488521#comment-13488521 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
bq. In other words, we should implement 'expected behavior' not necessarily what is convenient if there is a collision. (But convenience can always be provided via an option...) So if cygwin/sua show with spaces by default, that's what we should do. If they provide an option that puts separators in place, that's what we should. Consistency is key for (inter)operability.
Thanks Allen for commenting. We have generally tried to keep the consistency for expected behavior. For example, {{winutils ls}} will by default use the same format as Cygwin {{ls}} (with spaces). Given that this does not work for all cases on Windows, we introduced a flag that gives us the functionality we need. I honestly tried to find an existing flag when I first implemented this, but couldn't find one. It is totally possible that I missed it, please let me know if this is the case.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483827#comment-13483827 ]
Vinod Kumar Vavilapalli commented on HADOOP-8456:
-------------------------------------------------
AccessControlList depends on a space delimiter, you will need to somehow address that too.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499951#comment-13499951 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
Thanks Daryn and Suresh.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Fix For: 1-win
>
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484145#comment-13484145 ]
Daryn Sharp commented on HADOOP-8456:
-------------------------------------
Two questions:
# Does the wrapping of double quotes around the command args imply that the windows shell is re-parsing the group name? If so, that would seem to be a security issue if the user is somehow able to inject input with metachars.
# Maybe turn the delimiter into a general option so it's not hardcoded throughout the patch?
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497138#comment-13497138 ]
Daryn Sharp commented on HADOOP-8456:
-------------------------------------
Only suggestion would be to use {{getopt}} to process the args to simplify the logic, otherwise I think it looks good.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names.
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8456:
-------------------------------
Target Version/s: 1-win
Affects Version/s: (was: 0.24.0)
(was: 1.1.0)
1-win
> Support spaces in user names and group names.
> ---------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496886#comment-13496886 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
Allen, Daryn, Vinod, please let me know if you have any additional feedback on the patch.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8456:
-------------------------------
Attachment: HADOOP-8456.branch-1-win.spaces.2.patch
bq. Maybe turn the delimiter into a general option so it's not hardcoded throughout the patch?
Fixed in the next iteration. Let me know if it looks good.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names.
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8456:
-------------------------------
Attachment: HADOOP-8456.branch-1-win.spaces.patch
Attaching the patch.
> Support spaces in user names and group names.
> ---------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486234#comment-13486234 ]
Allen Wittenauer commented on HADOOP-8456:
------------------------------------------
bq. Generally, we want to natively integrate Hadoop with Windows, in a sense that Windows user/group information flows thru Hadoop.
bq. Let me know if this makes sense.
It does, but I'm just trying to get a measure for what the user expectation/experience is going to be for those that are going to use this interface. (Which, I suspect, is what Daryn's questions are about as well). 'ls' is a UNIXy thing, which why I asked specifically about cygwin and SUA. If Windows users are expecting different output, it may be worthwhile to implement a dir subcommand.
In other words, we should implement 'expected behavior' not necessarily what is convenient if there is a collision. (But convenience can always be provided via an option...) So if cygwin/sua show with spaces by default, that's what we should do. If they provide an option that puts separators in place, that's what we should. Consistency is key for (inter)operability.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8456) Support spaces in user names and
group names.
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8456:
-------------------------------
Status: Patch Available (was: Open)
> Support spaces in user names and group names.
> ---------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484182#comment-13484182 ]
Allen Wittenauer commented on HADOOP-8456:
------------------------------------------
How does Cygwin, MS SUA, etc handle this problem? We should do the same thing on Windows in order maintain compatibility with those systems.
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8456) Support spaces in user names and
group names in results returned via winutils
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498602#comment-13498602 ]
Ivan Mitic commented on HADOOP-8456:
------------------------------------
bq. Only suggestion would be to use getopt to process the args to simplify the logic, otherwise I think it looks good.
Thanks Daryn. If I understood your comment correctly, you are referring to [GNU C lib function|http://www.gnu.org/software/libc/manual/html_node/Using-Getopt.html#Using-Getopt] which does not exist in the C runtime lib we are using on Windows :)
> Support spaces in user names and group names in results returned via winutils
> -----------------------------------------------------------------------------
>
> Key: HADOOP-8456
> URL: https://issues.apache.org/jira/browse/HADOOP-8456
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 1-win
> Reporter: Chuan Liu
> Assignee: Ivan Mitic
> Priority: Minor
> Attachments: HADOOP-8456.branch-1-win.spaces.2.patch, HADOOP-8456.branch-1-win.spaces.patch
>
>
> When parsing results returned by ‘ls’, we made implicit assumption that user and group names cannot contain spaces. On Linux, spaces are not allowed in user names and group names. This is not the case for Windows. We need to find a way to fix the problem for Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira