You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sanjay Radia (Commented) (JIRA)" <ji...@apache.org> on 2012/02/13 18:56:59 UTC
[jira] [Commented] (MAPREDUCE-3825) MR should not be getting
duplicate tokens for a MR Job.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207011#comment-13207011 ]
Sanjay Radia commented on MAPREDUCE-3825:
-----------------------------------------
@Daryn> By not backwards-compatible, I'm referring to the fact that external filesystems,.....
Good point but the new apis in 23 or the new APIs you suggested will not be implemented by the external file systems and there is no default impl that works except for "not implemented" exception.
> MR should not be getting duplicate tokens for a MR Job.
> -------------------------------------------------------
>
> Key: MAPREDUCE-3825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3825.patch, TokenCache.pdf
>
>
> This is the counterpart to HADOOP-7967.
> MR gets tokens for all input, output and the default filesystem when a MR job is submitted.
> The APIs in FileSystem make it challenging to avoid duplicate tokens when there are file systems that have embedded
> filesystems.
> Here is the original description that Daryn wrote:
> The token cache currently tries to assume a filesystem's token service key. The assumption generally worked while there was a one to one mapping of filesystem to token. With the advent of multi-token filesystems like viewfs, the token cache will try to use a service key (ie. for viewfs) that will never exist (because it really gets the mounted fs tokens).
> The descriop
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira