[jira] Commented: (MAPREDUCE-1186) While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir

["Vinod K V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773828#action_12773828 ] 

Vinod K V commented on MAPREDUCE-1186:
--------------------------------------

Some attempts that I've verified which use DistributedCache got stuck doing chmod on the base-dir for the distcache. When I myself tried to replicate this scenario, I found out that chmod indeed takes a lot of time when number of files is large (For 35K files,2.7GB in archive directory, it took 1.2 minutes.) This should be fixed, we shouldn't do a recursive chmod on the whole archive directory for every task.

> While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1186
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1186
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>             Fix For: 0.21.0
>
>
> This is a performance problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.