You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Michael Gummelt (JIRA)" <ji...@apache.org> on 2017/04/13 23:38:41 UTC
[jira] [Created] (MAPREDUCE-6876) FileInputFormat.listStatus should
not fetch delegation tokens
Michael Gummelt created MAPREDUCE-6876:
------------------------------------------
Summary: FileInputFormat.listStatus should not fetch delegation tokens
Key: MAPREDUCE-6876
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6876
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Michael Gummelt
{{FileInputFormat.listStatus}} fetches delegation tokens: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java#L213
AFAICT, this is unnecessary. {{listStatus}} doesn't delegate those tokens to another process. This is causing issues described in the attached Spark Kerberos ticket, because {{TokenCache.obtainTokensForNameNodes}}, which is used to fetch the delegation tokens, assumes that certain MapReduce configuration variables are set, which isn't true in the Spark calling code. This is a separate problem, but nonetheless it wouldn't have arisen if {{listStatus}} weren't fetching delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org