[jira] [Created] (TEZ-1022) MRInputSplitDistributor does not use Vertex::numTasks when generating event payloads

["Hitesh Shah (JIRA)" <ji...@apache.org>]

Hitesh Shah created TEZ-1022:
--------------------------------

             Summary: MRInputSplitDistributor does not use Vertex::numTasks when generating event payloads
                 Key: TEZ-1022
                 URL: https://issues.apache.org/jira/browse/TEZ-1022
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Hitesh Shah


When the splits distributor and vertex parallelism are not in sync, it ends up 
creating events for non existent tasks causing:

2014-04-05 12:47:40,490 INFO [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.TaskImpl: task_1396727252029_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED
2014-04-05 12:47:40,491 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.NullPointerException
	at org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher.handle(DAGAppMaster.java:1670)
	at org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher.handle(DAGAppMaster.java:1656)
	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134)
	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81)
	at java.lang.Thread.run(Thread.java:722)
2014-04-05 12:47:40,495 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..




--
This message was sent by Atlassian JIRA
(v6.2#6252)