[jira] [Created] (MAPREDUCE-4843) When using DefaultTaskController, JobLocalizer not thread safe

["zhaoyunjiong (JIRA)" <ji...@apache.org>]

zhaoyunjiong created MAPREDUCE-4843:
---------------------------------------

             Summary: When using DefaultTaskController, JobLocalizer not thread safe
                 Key: MAPREDUCE-4843
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4843
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
    Affects Versions: 1.1.1
            Reporter: zhaoyunjiong
            Priority: Critical


In our cluster, some times job will failed due to below exception:
Error initializing attempt_201210181806_18566_r_000376_0: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/$username/jobcache/job_201210181806_18566/job.xml in any of the configured local directories at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:424)

The root cause is JobLocalizer is not thread safe.
In DefaultTaskController.initializeJob method:
     JobLocalizer localizer = new JobLocalizer((JobConf)getConf(), user, jobid);
but in JobLocalizer, it just simply keep the reference of the conf.
When two TaskLauncher threads(mapLauncher and reduceLauncher) try to initializeJob at same time, it will have two JobLocalizer, but one conf instance.
So some times ttConf.setStrings(JOB_LOCAL_CTXT, localDirs) will reset previous job's conf.
It will cause the previous job's job.xml stored at another user's dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira