[jira] [Commented] (TEZ-1370) Provide a configuration so when a session is started a set of containers are started with it

["Thaddeus Diamond (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084123#comment-14084123 ] 

Thaddeus Diamond commented on TEZ-1370:
---------------------------------------

It seems like the only place Tez requests containers is in the {{TezAMRMClientAsync}} (called by the {{YarnTaskSchedulerService}}), which, if you unwind the stack, is only triggered by a DAG submission event.  Is there any way to fake this?  Or could we submit a DAG with a guaranteed number of containers?

> Provide a configuration so when a session is started a set of containers are started with it
> --------------------------------------------------------------------------------------------
>
>                 Key: TEZ-1370
>                 URL: https://issues.apache.org/jira/browse/TEZ-1370
>             Project: Apache Tez
>          Issue Type: Wish
>    Affects Versions: 0.5.0
>         Environment: branch -> master
> hadoop -> HDP 2.1/CDH 5.0
>            Reporter: Thaddeus Diamond
>
> This could use the existing {{TezConfiguration.TEZ_AM_SESSION_MIN_HELD_CONTAINERS}} configuration or a new one.
> The API would be as follows:
> {noformat}
> TezConfiguration tezConfiguration = new TezConfiguration(hadoopConfiguration);
> tezConfiguration.setBoolean(TEZ_AM_SESSION_MODE, true)
> tezConfiguration.setInt(TEZ_AM_SESSION_STARTUP_CONTAINERS, 3)
> TezClient tezClient = new TezClient("foobar", tezConfiguration);
> // The following should start 3 containers
> tezClient.start();
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)