You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Hitesh Shah (JIRA)" <ji...@apache.org> on 2014/08/29 22:29:52 UTC
[jira] [Commented] (TEZ-1524) getDAGStatus seems to fork out the
entire JVM
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115769#comment-14115769 ]
Hitesh Shah commented on TEZ-1524:
----------------------------------
UGI should be using the Groups class that does caching internally.
Can you look for these logs in debug mode ( looking at 2.6.0-SNAPSHOT code though ):
- "Returning cached groups for" ( implies obtained from cache )
- "Returning fetched groups"
> getDAGStatus seems to fork out the entire JVM
> ---------------------------------------------
>
> Key: TEZ-1524
> URL: https://issues.apache.org/jira/browse/TEZ-1524
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Gopal V
> Fix For: 0.6.0
>
>
> Tracked down a consistent fork() call to
> {code}
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
> at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1409)
> at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getRPCUserGroups(DAGClientAMProtocolBlockingPBServerImpl.java:75)
> at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:102)
> at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> {code}
> [~hitesh] - would it make sense to cache this at all?
--
This message was sent by Atlassian JIRA
(v6.2#6252)