You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ivan Mitic (JIRA)" <ji...@apache.org> on 2012/09/06 16:29:07 UTC
[jira] [Commented] (HADOOP-8734) LocalJobRunner does not support
private distributed cache
[ https://issues.apache.org/jira/browse/HADOOP-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449703#comment-13449703 ]
Ivan Mitic commented on HADOOP-8734:
------------------------------------
Thanks for reviewing Bikas.
bq.Can you please elaborate on the cause and the fix?
The problem I see is that LocalJobRunner does not work with private distributed cache files. Private DC files are files that are not visible to "others" on the LFS leading to them being marked as private in DC. Check out the fix I did to TestMRWithDistributedCache, this is an E2E use case. Specifically, {{testLocalJobRunner}} now fails without my code fix.
The fix is in the LocalJobRunner to explicitly localize the private cache (I saw this pattern being used for private DC cache localization in some tests).
This is something that is nice to have especially on Windows, as all files are private by default (not visible to "others"). HADOOP-8731 has some info on DC on Windows that might be helpful.
> LocalJobRunner does not support private distributed cache
> ---------------------------------------------------------
>
> Key: HADOOP-8734
> URL: https://issues.apache.org/jira/browse/HADOOP-8734
> Project: Hadoop Common
> Issue Type: Bug
> Components: filecache
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8734-LocalJobRunner.patch
>
>
> It seems that LocalJobRunner does not support private distributed cache. The issue is more visible on Windows as all DC files are private by default (see HADOOP-8731).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira