[jira] [Updated] (TEZ-1592) Vertex should wait for all initializers to finish before moving to INITED state

["Siddharth Seth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated TEZ-1592:
--------------------------------
    Attachment: TEZ-1592.1.txt

After the patch, the Vertex waits till all initializers are complete before moving into INITED state. Also consolidated the code for setting up initializers, which was mostly duplicated, and missing initWaitsForRootInitializers in one case.

All initializers must complete before we move into INITED state. That isn't always a requirement - but at the moment we don't know when not to wait for an initializer, since any of them could lead to parallelism being updated.

[~bikassaha] - pleas review.

> Vertex should wait for all initializers to finish before moving to INITED state
> -------------------------------------------------------------------------------
>
>                 Key: TEZ-1592
>                 URL: https://issues.apache.org/jira/browse/TEZ-1592
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>            Priority: Critical
>         Attachments: TEZ-1592.1.txt
>
>
> Reported by [~vikram.dixit]
> When using multiple initializers, the following stack trace is seen at times.
> {code}
> 2014-09-17 15:05:00,406 ERROR [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.VertexImpl: Can't handle Invalid event V_ROOT_INPUT_INITIALIZED on vertex Map 2 with vertexId vertex_1410991351910_0002_8_01 at current state RUNNING
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_ROOT_INPUT_INITIALIZED at RUNNING
>   at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
>   at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
>   at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
>   at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1337)
>   at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:168)
>   at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1641)
>   at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1627)
>   at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
>   at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
>   at java.lang.Thread.run(Thread.java:662)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)