You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2012/11/27 20:01:59 UTC
[jira] [Created] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Jason Lowe created MAPREDUCE-4825:
-------------------------------------
Summary: JobImpl.finished doesn't expect ERROR as a final job state
Key: MAPREDUCE-4825
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Reporter: Jason Lowe
TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
{noformat}
2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
java.lang.IllegalArgumentException: Illegal job state: ERROR
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
at java.lang.Thread.run(Thread.java:662)
2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506527#comment-13506527 ]
Jason Lowe commented on MAPREDUCE-4825:
---------------------------------------
bq. Would there be a problem in metrics being notified of success/failure and then again of error?
Potentially, I forgot the job could leave these terminal states. Some potential ways to address it:
* Don't allow the state to leave "terminal" states like SUCCEEDED/FAILED/KILLED.
* Add metrics for "errored" jobs to distinguish between failed and error. This still means that the sum of metrics could exceed the total number of job since a job can both succeed and error.
* Have finished ignore incrementing any metrics if the job is already in a terminal state (SUCCEEDED/FAILED/KILLED) to avoid double-counting a job.
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505730#comment-13505730 ]
Hudson commented on MAPREDUCE-4825:
-----------------------------------
Integrated in Hadoop-trunk-Commit #3069 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3069/])
MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) (Revision 1414840)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414840
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4825:
----------------------------------
Assignee: Jason Lowe
Target Version/s: 2.0.3-alpha, 0.23.6
Affects Version/s: 0.23.5
2.0.3-alpha
Status: Patch Available (was: Open)
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504861#comment-13504861 ]
Hadoop QA commented on MAPREDUCE-4825:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12555049/MAPREDUCE-4825.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3072//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3072//console
This message is automatically generated.
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505701#comment-13505701 ]
Robert Joseph Evans commented on MAPREDUCE-4825:
------------------------------------------------
The patch looks fine to me. +1
I'll check it in.
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506264#comment-13506264 ]
Bikas Saha commented on MAPREDUCE-4825:
---------------------------------------
The JobImpl state machine allows SUCCEEDED->ERROR and FAILED->ERROR transition which would call finished(). Would there be a problem in metrics being notified of success/failure and then again of error?
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506457#comment-13506457 ]
Hudson commented on MAPREDUCE-4825:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1241 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1241/])
MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) (Revision 1414840)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414840
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506440#comment-13506440 ]
Hudson commented on MAPREDUCE-4825:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #450 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/450/])
svn merge -c 1414840 FIXES: MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) (Revision 1414844)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414844
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506389#comment-13506389 ]
Hudson commented on MAPREDUCE-4825:
-----------------------------------
Integrated in Hadoop-Yarn-trunk #51 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/51/])
MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) (Revision 1414840)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414840
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4825:
-------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.6
2.0.3-alpha
3.0.0
Status: Resolved (was: Patch Available)
Thanks Jason,
I put this in trunk, branch-2, and branch-0.23
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4825:
----------------------------------
Attachment: MAPREDUCE-4825.patch
Simple fix. No additional unit tests since this is fixing an existing test.
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Reporter: Jason Lowe
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4825) JobImpl.finished doesn't expect
ERROR as a final job state
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507373#comment-13507373 ]
Jason Lowe commented on MAPREDUCE-4825:
---------------------------------------
Filed MAPREDUCE-4835 to track the metric double-counting problem. Thanks for pointing out the issue, Bikas!
> JobImpl.finished doesn't expect ERROR as a final job state
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: MAPREDUCE-4825.patch
>
>
> TestMRApp.testJobError is causing AsyncDispatcher to exit with System.exit due to an exception being thrown. From the console output from testJobError:
> {noformat}
> 2012-11-27 18:46:15,240 ERROR [AsyncDispatcher event handler] impl.TaskImpl (TaskImpl.java:internalError(665)) - Invalid event T_SCHEDULE on Task task_0_0000_m_000000
> 2012-11-27 18:46:15,242 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(132)) - Error in dispatcher thread
> java.lang.IllegalArgumentException: Illegal job state: ERROR
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.finished(JobImpl.java:838)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1622)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InternalErrorTransition.transition(JobImpl.java:1)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$3(StateMachineFactory.java:287)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:723)
> at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:974)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2012-11-27 18:46:15,242 INFO [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(135)) - Exiting, bbye..
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira