[jira] [Commented] (MAPREDUCE-4396) Make LocalJobRunner work with private distributed cache

["Yu Gao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492762#comment-13492762 ] 

Yu Gao commented on MAPREDUCE-4396:
-----------------------------------

Attached ant test and test-patch result.

This patch can be verified by running unit test TestMRWithDistributedCache in a user directory with restrictive permission (e.g. 700), so no new test is added.

Before this patch, TestMRWithDistributedCache#testLocalJobRunner could fail when the test directory - test.build.data - is in a path where any ancestor doesn't have +x permissions for others, with following exception:
java.io.FileNotFoundException: File build/test/mapred/local/archive/5659425182264546915_1878220893_1556512037/file/home/biadmin/Hadoop_upstream/branch-1/build/test/data/distributed.first does not exist.
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
	at org.apache.hadoop.filecache.TestMRWithDistributedCache$DistributedCacheChecker.setup(TestMRWithDistributedCache.java:93)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

This results from private distributed cache not localized by LocalJobRunner. The patch adds the localization for private distributed cache, so that no matter whether the test directory permission is too restrictive or not, distributed cache (public/private) will be localized.
                
> Make LocalJobRunner work with private distributed cache
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-4396
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4396
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 1.0.3
>            Reporter: Luke Lu
>            Assignee: Yu Gao
>            Priority: Minor
>         Attachments: mapreduce-4396-branch-1.patch, test-afterpatch.result, test-beforepatch.result, test-patch.result
>
>
> Some LocalJobRunner related unit tests fails if user directory permission and/or umask is too restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira