You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2011/01/21 00:21:44 UTC
[jira] Created: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Add a cache for getpwuid_r and getpwgid_r calls
-----------------------------------------------
Key: HADOOP-7115
URL: https://issues.apache.org/jira/browse/HADOOP-7115
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Arun C Murthy
Assignee: Devaraj Das
As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492717#comment-13492717 ]
Hadoop QA commented on HADOOP-7115:
-----------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552538/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1719//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1719//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985078#action_12985078 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
bq. Do we really want to set a precedent where we start patching Hadoop due to specific vendor's implementation problems
Yes. In my opinion this is unavoidable for large scale systems.
But I agree with Koji that these workarounds should be configurable and probably off-by-default.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985048#action_12985048 ]
Koji Noguchi commented on HADOOP-7115:
--------------------------------------
Devaraj, can we make this cache timeout period configurable ?
If yes, ops would have an option to set this timeout to 0 to skip the caching altogether.
Allen, will this address your concern?
We really had miserable weeks when our LDAP servers were going up and down. Don't want to go back there.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492770#comment-13492770 ]
Hadoop QA commented on HADOOP-7115:
-----------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552549/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1721//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1721//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur reassigned HADOOP-7115:
------------------------------------------
Assignee: Alejandro Abdelnur (was: Devaraj Das)
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289263#comment-13289263 ]
Hudson commented on HADOOP-7115:
--------------------------------
Integrated in Hadoop-Common-22-branch #106 (See [https://builds.apache.org/job/Hadoop-Common-22-branch/106/])
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls. Contributed by Devaraj Das and Benoy Antony. (Revision 1346246)
Result = SUCCESS
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346246
Files :
* /hadoop/common/branches/branch-0.22/common/CHANGES.txt
* /hadoop/common/branches/branch-0.22/common/src/java/core-default.xml
* /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/io/SecureIOUtils.java
* /hadoop/common/branches/branch-0.22/common/src/java/org/apache/hadoop/io/nativeio/NativeIO.java
* /hadoop/common/branches/branch-0.22/common/src/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/branches/branch-0.22/common/src/native/src/org/apache/hadoop/security/getGroup.c
* /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/io/TestSecureIOUtils.java
* /hadoop/common/branches/branch-0.22/common/src/test/core/org/apache/hadoop/io/nativeio/TestNativeIO.java
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Fix For: 0.22.1
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-7115:
----------------------------------------
Fix Version/s: 0.22.1
I just committed this to branch 0.22.1. Thank you Deveraj and Benoy.
Will close it if not needed in other version.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Fix For: 0.22.1
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494616#comment-13494616 ]
Hudson commented on HADOOP-7115:
--------------------------------
Integrated in Hadoop-Yarn-trunk #32 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/32/])
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls (tucu) (Revision 1407577)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1407577
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985052#action_12985052 ]
Allen Wittenauer commented on HADOOP-7115:
------------------------------------------
Do we really want to set a precedent where we start patching Hadoop due to specific vendor's implementation problems? That's where we are headed with this patch....
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492733#comment-13492733 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
- why is cacheTimeout volatile? You're already synchronizing on the class inside {{ensureInitialized}}.
- instead of {{ensureInitialized}}, why not initialize the configuration in the existing static {...} block?
- instead of using ints for {{USER}} and {{GROUP}}, how about an enum like this:
{code}
enum IdCache {
USERNAME,
GROUP;
ConcurrentHashMap<Integer, CachedName> cache =
new ConcurrentHashMap<Integer, CachedName>();
}
...
private static String getName(IdCache cache, int id) {
CachedName cachedName = cache.cache.get(id);
...
}
{code}
this way you get typesafety, and you can just stringify the {{IdCache}} instance to get a printable name
Otherwise looks good to me
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Benoy Antony (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoy Antony updated HADOOP-7115:
---------------------------------
Attachment: hadoop-7115-0.22.patch
Reattaching the patch after removing the changes for mapreduce. The changes for mapreduce are attached in MAPREDUCE-2353
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494666#comment-13494666 ]
Hudson commented on HADOOP-7115:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #1222 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1222/])
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls (tucu) (Revision 1407577)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1407577
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12984886#action_12984886 ]
Jakob Homan commented on HADOOP-7115:
-------------------------------------
In production we found nscd to be very unreliable, causing storms against the LDAP server when it goes down.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985178#action_12985178 ]
Allen Wittenauer commented on HADOOP-7115:
------------------------------------------
> Yes. In my opinion this is unavoidable for large scale systems.
>From my understanding, the reliability issue was fixed by upgrading libc and/or nscd. This makes it an OS bug that was fixed with a vendor supplied patch. We need to be very clear on what we want to do when it comes to OS workarounds. Right now, we're incredibly inconsistent. (and yes, I'm thinking specifically of the whoami, getent, etc problems.)
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985023#action_12985023 ]
Allen Wittenauer commented on HADOOP-7115:
------------------------------------------
Do we actually know it was getpwuid or some other call? Yahoo uses a lot of netgroup info--which is not cached by nscd. The other thing is that nscd is tunable. You might have just needed to bump up the size of the ncsd cache.
Given how previous "caches" have been implemented, I'm extremely hesitant to see another one implemented. Especially one that would likely require ops to bring the whole grid down to fix. [It is bad enough we have to bounce the NN/JT to fix some of these caches.]
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001616#comment-13001616 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
Hey Devaraj. Any news on this patch for trunk? I need to fix HADOOP-7156 which is likely to conflict with this code.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003835#comment-13003835 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
A few comments:
- getUIDforFDOwnerforOwner is a kind of messy name - maybe just getUIDForFDOwner?
- The following should probably be debug level:
{code}
LOG.info("Got UserName " + user + " for UID " + uid + " from the native implementation");
{code}
- ensureInitialized has a formatting issue (cacheTimeout is on the same line as the opening if)
- The log message there should also probably be debug level
- Do we need fstat and the Stat type at all now? I think not, we might as well remove it.
- The change in getGroup.c seems unrelated to this JIRA, right?
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Attachments: h-7115.1.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Fix Version/s: 2.0.3-alpha
Status: Patch Available (was: Open)
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.4, 2.0.2-alpha, 0.22.0
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Attachment: HADOOP-7115.patch
Thanks for the review Todd, new patch addressing you comments. Minor tweak, converted the INT constants to an ENUM but did not move the cache into the ENUM, left as before (not very keen of mutable ENUMs).
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12984882#action_12984882 ]
Allen Wittenauer commented on HADOOP-7115:
------------------------------------------
This is what nscd should be providing for you at the OS level. Why do we want another cache?
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494677#comment-13494677 ]
Hudson commented on HADOOP-7115:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1253 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1253/])
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls (tucu) (Revision 1407577)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1407577
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Benoy Antony (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoy Antony updated HADOOP-7115:
---------------------------------
Attachment: hadoop-7115-0.22.patch
Patch for 22
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Attachment: HADOOP-7115.patch
New patch addressing Todd's last 2 comments
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492754#comment-13492754 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
{code}
+ cacheTimeout = new Configuration().getLong(
+ CommonConfigurationKeys.HADOOP_SECURITY_UID_NAME_CACHE_TIMEOUT_KEY,
+ CommonConfigurationKeys.HADOOP_SECURITY_UID_NAME_CACHE_TIMEOUT_DEFAULT) *
+ 1000;
+ LOG.debug("Initialized cache for IDs to User/Group mapping with a" +
+ " cache timeout of " + cacheTimeout/1000 + " seconds.");
{code}
This should move up into the try block above -- if the native library fails to load, then it shouldn't log this debug message (since the cache won't ever be used). Additionally you can reuse the Configuration instance that's already constructed above.
- Just noticed an addition of {{syslog.h}} to the native code - doesn't seem to be used.
Otherwise looks good.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Attachment: HADOOP-7115.patch
Patch for trunk. It revives the cache for ID->NAME. It has been refactored a bit to be used for both UIDs and GIDs. It has been tested in a secure cluster configured to use and external directory for user provisioning running terasort jobs. Without this patch the same cluster cannot run a single terasort job for non-local users.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985042#action_12985042 ]
Devaraj Das commented on HADOOP-7115:
-------------------------------------
It was the getpwuid calls leading to ldap outages in our major clusters (almost twice every week). After implementing this cache, we didn't have a single incident (and hopefully we will have none in the future too).
The version of nscd on our clusters proved to be really unreliable.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494118#comment-13494118 ]
Hadoop QA commented on HADOOP-7115:
-----------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552849/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1725//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1725//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491972#comment-13491972 ]
Alejandro Abdelnur commented on HADOOP-7115:
--------------------------------------------
Just chatted with Devaraj and he is OK with me taking over this JIRA to have a patch for trunk. THX
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003839#comment-13003839 ]
Todd Lipcon commented on HADOOP-7115:
-------------------------------------
bq. Do we need fstat and the Stat type at all now? I think not, we might as well remove it.
Actually, just to keep this closer to the 0.20 security branch, why don't we do that part in a separate JIRA. Filed HADOOP-7173.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Attachments: h-7115.1.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Attachment: HADOOP-7115.patch
minor udpate, adding private/unstable annotations to the NativeIO.java class
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492747#comment-13492747 ]
Alejandro Abdelnur commented on HADOOP-7115:
--------------------------------------------
Forgot the mention before, the native part of the patch it was done by Sandy Ryza (thx Sandy).
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494164#comment-13494164 ]
Tom White commented on HADOOP-7115:
-----------------------------------
+1 the latest patch looks good to me.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494106#comment-13494106 ]
Hadoop QA commented on HADOOP-7115:
-----------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552849/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1724//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1724//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-7115:
--------------------------------
Target Version/s: 3.0.0, 2.0.3-alpha
Affects Version/s: 2.0.2-alpha
0.23.4
We need this in trunk, branch-2, and probably branch-0.23, too - not sure how it got missed somewhere along the line
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Fix For: 0.22.1
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494190#comment-13494190 ]
Hudson commented on HADOOP-7115:
--------------------------------
Integrated in Hadoop-trunk-Commit #2992 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2992/])
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls (tucu) (Revision 1407577)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1407577
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7115:
---------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
committed to trunk and branch-2.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
> Reporter: Arun C Murthy
> Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-7115) Add a cache for getpwuid_r and
getpwgid_r calls
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-7115:
--------------------------------
Attachment: h-7115.1.patch
This patch is a straight port from the 20.100 security branch to trunk for the corresponding commit in the security branch. I will also raise a MR jira to address the MR side of the changes.
> Add a cache for getpwuid_r and getpwgid_r calls
> -----------------------------------------------
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Arun C Murthy
> Assignee: Devaraj Das
> Attachments: h-7115.1.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira