[jira] Commented: (MAPREDUCE-1288) DistributedCache localizes only once per cache URI

["Vinod K V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892724#action_12892724 ] 

Vinod K V commented on MAPREDUCE-1288:
--------------------------------------

Devaraj, this corner case is exactly what Hemanth was trying to explain earlier on this ticket, starting with comment #4 above :)

As for the approach, we have two options:
 (1) (this seems to be what the patch is doing) for group shared files, localize them separately for each user. This is a simple solution, but sacrifices the optimization ( may not be too bad?)
 (2) introduce the concept of group sharing of distributed cache files so as to avoid repetitive downloads for group shared files also. This may be a complex solution after all.

> DistributedCache localizes only once per cache URI
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1288
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1288
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distributed-cache, security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Devaraj Das
>            Priority: Critical
>         Attachments: MR-1288-bp20-1.patch, MR-1288-bp20-2.patch
>
>
> As part of the file localization the distributed cache localizer creates a copy of the file in the corresponding user's private directory. The localization in DistributedCache assumes the key as the URI of the cachefile and if it already exists in the map, the localization is not done again. This means that another user cannot access the same distributed cache file. We should change the key to include the username so that localization is done for every user.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.