[jira] [Created] (TEZ-2609) Consider calling TaskReporter.heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask

["Rajesh Balamohan (JIRA)" <ji...@apache.org>]

Rajesh Balamohan created TEZ-2609:
-------------------------------------

             Summary: Consider calling TaskReporter.heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask
                 Key: TEZ-2609
                 URL: https://issues.apache.org/jira/browse/TEZ-2609
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan


Default TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS is around 100 ms.  This works for most of the usecases. However, for large jobs (10000s of tasks), this can be a problem and have timeout issues. Setting this to a very large value would degrade the job runtime and lower value can cause timeout issues for large jobs.

It might be worth to consider deferring heartbeat() when there are events to be processed in LogicalIOProcessorRuntimeTask.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)