Communication between YARN and Tez.

[alokasok <al...@buffalo.edu>]

Hi,

I was trying to get an idea about how the communication is handled 
between Tez and Yarn for task completion and task submission.Is there 
any documentation in this regards to this other than the java doc for 
the project.


Regards,

Alok Asok

Re: Communication between YARN and Tez.

["Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>]

Hi Alok,

Tez is one kind of yarn application. So Tez use the standard yarn protocol
to communicate with RM & NN (submit/register/unregister app & launch
containers)
Regarding your question on the task completion & task submission. This is
a tez propriety protocol TezTaskUmbilicalProtocol between tez AM
(DAGAppMaster) and tez workers (TezChild)

Paste the code as below,

Currently
  1) TezChild pull task from Tez AM (TezTaskUmbilicalProtocol.getTask)
  2) Tez AM get the task status from TezChild through periodic heartbeat
(TezTaskUmbilicalProtocol.heartbeat)


public interface TezTaskUmbilicalProtocol extends VersionedProtocol {

  public static final long versionID = 19L;

  ContainerTask getTask(ContainerContext containerContext) throws
IOException;

  boolean canCommit(TezTaskAttemptID taskid) throws IOException;

  public TezHeartbeatResponse heartbeat(TezHeartbeatRequest request)
      throws IOException, TezException;

}

Hope it helps you.





Best Regard,
Jeff Zhang





On 3/25/15, 4:11 PM, "alokasok" <al...@buffalo.edu> wrote:

>Hi,
>
>I was trying to get an idea about how the communication is handled
>between Tez and Yarn for task completion and task submission.Is there
>any documentation in this regards to this other than the java doc for
>the project.
>
>
>Regards,
>
>Alok Asok
>
>