[jira] [Commented] (TEZ-4010) Use InputReadyVertexManager for broadcast connection

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653140#comment-16653140 ] 

Gopal V commented on TEZ-4010:
------------------------------

bq. This seems to be inefficient as tasks are started way before they can run in the DAG.

This made sense a long time ago when the BI queries need to start the YARN containers ahead of the broadcast tasks (the YARN task spinups take between 3-10 seconds).

At this point (with LLAP replacing YARN containers for BI), I'd say that this is an appendix to be removed and replaced with the InputReadyVertexManager instead.

> Use InputReadyVertexManager for broadcast connection
> ----------------------------------------------------
>
>                 Key: TEZ-4010
>                 URL: https://issues.apache.org/jira/browse/TEZ-4010
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Hitesh Sharma
>            Priority: Minor
>
> As per [VertexImpl::assignVertexManager|https://github.com/apache/tez/blob/39d76a656216d4843908279ef8eaa29a4cc83104/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java#L2677] an ImmediateStartVertexManager is used for broadcast connections. This seems to be inefficient as tasks are started way before they can run in the DAG.
> Thoughts on using InputReadyVertexManager for broadcast connections?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)