[jira] [Commented] (MAPREDUCE-5513) ConcurrentModificationException in JobControl

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770815#comment-13770815 ] 

Jason Lowe commented on MAPREDUCE-5513:
---------------------------------------

Sample backtrace when this occurs in a Pig job:

{noformat}
org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1286)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1260)
    at org.apache.pig.PigServer.execute(PigServer.java:1250)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:435)
    at org.apache.pig.PigRunner.run(PigRunner.java:49)
    at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
    at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:494)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:400)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
Caused by: java.util.ConcurrentModificationException
    at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:953)
    at java.util.LinkedList$ListItr.next(LinkedList.java:886)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.toList(JobControl.java:83)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.getSuccessfulJobList(JobControl.java:125)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.getSuccessfulJobs(JobControl.java:75)
    at org.apache.pig.tools.pigstats.PigStatsUtil.accumulateStats(PigStatsUtil.java:244)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:304)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
    ... 26 more
{noformat}
                
> ConcurrentModificationException in JobControl
> ---------------------------------------------
>
>                 Key: MAPREDUCE-5513
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5513
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta, 0.23.9
>            Reporter: Jason Lowe
>
> JobControl.toList is locking individual lists to iterate them, but those lists can be modified elsewhere without holding the list lock.  The locking approaches are mismatched, with toList holding the lock on the actual list object while other methods hold the JobControl lock when modifying the lists.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira