[jira] [Comment Edited] (TEZ-692) Unify job submission in either TezClient or TezSession

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052065#comment-14052065 ] 

Bikas Saha edited comment on TEZ-692 at 7/4/14 1:09 AM:
--------------------------------------------------------

This is in an incompatible change. Steps to make code compile again.
1) TezSession is gone. TezClient is the only interface. Session/Non-session mode is determined via TezConfiguration tez.am.mode.session or hard-coded in the constructor. Recommendation is to not hard-code. If properly written, the code should be the same both ways and session/non-session can be decided via configuration when executing the job.
2) Remove AMConfiguration - If app master specific credentials/resources were being passed via AMConfiguration, then they need to be passed in via the new constructor.
3) start() needs to be called before submitting a DAG.
4) call waitTillReady() before submitting the DAG.
5) stop() needs to be called at the end to release connection to cluster, ideally in a finally block.
6) TezCient does not generate a YARN appId for the app master before submitting the DAG. The appId is available after DAG is submitted. This is best practice to handle forward looking HA scenarios when YARN will return the appId after app submission. Users relying on appId as a unique string should use some other method.

The new code should be a lot simpler and concise than earlier.
Non-session mode is the default. If you want to use sessions, then please set it in the conf or use the constructor to explicitly enable it.


was (Author: bikassaha):
This is in an incompatible change. Steps to make code compile again.
1) TezSession is gone. TezClient is the only interface. Session/Non-session mode is determined via TezConfiguration tez.am.mode.session or hard-coded in the constructor. Recommendation is to not hard-code. If properly written, the code should be the same both ways and session/non-session can be decided via configuration when executing the job.
2) Remove AMConfiguration - If app master specific credentials/resources were being passed via AMConfiguration, then they need to be passed in via the new constructor.
3) start() needs to be called before submitting a DAG.
4) call waitTillReady() before submitting the DAG.
5) stop() needs to be called at the end to release connection to cluster, ideally in a finally block.
6) TezCient does not generate a YARN appId for the app master before submitting the DAG. The appId is available after DAG is submitted. This is best practice to handle forward looking HA scenarios when YARN will return the appId after app submission. Users relying on appId as a unique string should use some other method.

The new code should be a lot simpler and concise than earlier.

> Unify job submission in either TezClient or TezSession
> ------------------------------------------------------
>
>                 Key: TEZ-692
>                 URL: https://issues.apache.org/jira/browse/TEZ-692
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-692.1.patch, TEZ-692.2.patch, TEZ-692.3.patch, TEZ-692.4.patch, TEZ-692.5.patch
>
>
> Its confusing to have 2 ways to create and submit a tez job. The developer has to spend time thinking about and deciding which method to use.



--
This message was sent by Atlassian JIRA
(v6.2#6252)