You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ali Kheradmand (JIRA)" <ji...@apache.org> on 2015/10/29 00:02:27 UTC
[jira] [Updated] (MAPREDUCE-6527) Data race on field
org.apache.hadoop.mapred.JobConf.credentials
[ https://issues.apache.org/jira/browse/MAPREDUCE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ali Kheradmand updated MAPREDUCE-6527:
--------------------------------------
Description:
I am running the test suite against a dynamic race detector called RV-Predict. Here is a race report that I got:
Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
Concurrent read in thread T327 (locks held: {})
----> at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
T327 is created by T22
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218)
Concurrent write in thread T22 (locks held: {Monitor@496c673a, Monitor@496319b0})
----> at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
locked Monitor@496319b0 at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)
at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
locked Monitor@496c673a at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)
T22 is created by T1
at org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
}}}
In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob function, we have the following lines:
Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
job.job.setCredentials(credentials);
It looks a bit suspicious: Job extends thread and at the end of its constructor it starts a new thread which creates a new instance of JobContextImpl which reads credentials. However, the first thread concurrently sets credentials after a creating the Job instance.
was:
I am running the test suite against a dynamic race detector called RV-Predict. Here is a race report that I got:
Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
Concurrent read in thread T327 (locks held: {})
----> at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
T327 is created by T22
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218)
Concurrent write in thread T22 (locks held: {Monitor@496c673a, Monitor@496319b0})
----> at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
- locked Monitor@496319b0 at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)
at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
- locked Monitor@496c673a at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)
T22 is created by T1
at org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
}}}
In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob function, we have the following lines:
Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
job.job.setCredentials(credentials);
It looks a bit suspicious: Job extends thread and at the end of its constructor it starts a new thread which creates a new instance of JobContextImpl which reads credentials. However, the first thread concurrently sets credentials after a creating the Job instance.
> Data race on field org.apache.hadoop.mapred.JobConf.credentials
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-6527
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6527
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Ali Kheradmand
>
> I am running the test suite against a dynamic race detector called RV-Predict. Here is a race report that I got:
> Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
> Concurrent read in thread T327 (locks held: {})
> ----> at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
> at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
> T327 is created by T22
> at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218)
> Concurrent write in thread T22 (locks held: {Monitor@496c673a, Monitor@496319b0})
> ----> at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
> at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> locked Monitor@496319b0 at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)
> at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
> locked Monitor@496c673a at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)
> T22 is created by T1
> at org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
> }}}
> In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob function, we have the following lines:
> Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
> job.job.setCredentials(credentials);
> It looks a bit suspicious: Job extends thread and at the end of its constructor it starts a new thread which creates a new instance of JobContextImpl which reads credentials. However, the first thread concurrently sets credentials after a creating the Job instance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)