You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2018/07/05 23:54:00 UTC
[jira] [Commented] (HDFS-13723) Occasional "Should be different
group" error in TestRefreshUserMappings#testGroupMappingRefresh
[ https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534299#comment-16534299 ]
Wei-Chiu Chuang commented on HDFS-13723:
----------------------------------------
Thanks [~smeng] for reporting the issue and the patch. Overall makes sense.
Essentially it relaxes the timeout constraint to accommodate potential slowness in cache reload thread.
Could you add "@VisibleForTesting" annotation above this line?
{code}
static final Logger LOG = LoggerFactory.getLogger(Groups.class);
{code}
This is just to mark that this variable is supposed to be private, and we make it package private only for testing purposes.
> Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
> Affects Versions: 3.0.0
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Attachments: HDFS-13723.001.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion would fail due to the mapping didn't refresh in time, reporting "Should be different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647 heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode (BPOfferService.java:processCommandFromActive(759)) - Got finalize command for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport src=null dst=null perm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport src=null dst=null perm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648)) - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=refreshUserToGroupsMappings src=null dst=null perm=null proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org