You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tez.apache.org by Robert Grandl <rg...@yahoo.com.INVALID> on 2016/01/12 04:10:58 UTC

Response from RM

Hi guys,
I am having a setup where I am running Tez 0.7.0 and Hadoop 2.6.0. 

I am trying to propagate some additional information from RM to Tez AM. To do that, I was modifying the heartbeat response datastructure from RM to introduce an additional field. I modified AllocareResponseProto datastructure from yarn_service_protos.proto and the inherent classes in Yarn. 

However, I could not find where in Tez the heartbeat response is received and hence where to get my additional information, propagated back from Yarn.

Thanks in advance,Robert

Re: Response from RM

Posted by Robert Grandl <rg...@yahoo.com.INVALID>.

Thanks Jeff. That was very helpful. 

Robert 

    On Monday, January 11, 2016 9:47 PM, Jianfeng (Jeff) Zhang <jz...@hortonworks.com> wrote:
 

 
Tez doesn't use the low level api, it use AMRMClientAsync which has shaded AllocateResponse.

https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/async/impl/AMRMClientAsyncImpl.java



Best Regard,
Jeff Zhang




From: Robert Grandl <rg...@yahoo.com>>
Reply-To: "user@tez.apache.org<ma...@tez.apache.org>" <us...@tez.apache.org>>, Robert Grandl <rg...@yahoo.com>>
Date: Monday, January 11, 2016 at 7:10 PM
To: User <us...@tez.incubator.apache.org>>, "dev@tez.incubator.apache.org<ma...@tez.incubator.apache.org>" <de...@tez.incubator.apache.org>>
Subject: Response from RM

Hi guys,

I am having a setup where I am running Tez 0.7.0 and Hadoop 2.6.0.

I am trying to propagate some additional information from RM to Tez AM. To do that, I was modifying the heartbeat response datastructure from RM to introduce an additional field. I modified AllocareResponseProto datastructure from yarn_service_protos.proto and the inherent classes in Yarn.

However, I could not find where in Tez the heartbeat response is received and hence where to get my additional information, propagated back from Yarn.

Thanks in advance,
Robert

Re: Response from RM

Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.

Tez doesn't use the low level api, it use AMRMClientAsync which has shaded AllocateResponse.

https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/async/impl/AMRMClientAsyncImpl.java



Best Regard,
Jeff Zhang


From: Robert Grandl <rg...@yahoo.com>>
Reply-To: "user@tez.apache.org<ma...@tez.apache.org>" <us...@tez.apache.org>>, Robert Grandl <rg...@yahoo.com>>
Date: Monday, January 11, 2016 at 7:10 PM
To: User <us...@tez.incubator.apache.org>>, "dev@tez.incubator.apache.org<ma...@tez.incubator.apache.org>" <de...@tez.incubator.apache.org>>
Subject: Response from RM

Hi guys,

I am having a setup where I am running Tez 0.7.0 and Hadoop 2.6.0.

I am trying to propagate some additional information from RM to Tez AM. To do that, I was modifying the heartbeat response datastructure from RM to introduce an additional field. I modified AllocareResponseProto datastructure from yarn_service_protos.proto and the inherent classes in Yarn.

However, I could not find where in Tez the heartbeat response is received and hence where to get my additional information, propagated back from Yarn.

Thanks in advance,
Robert

Re: Response from RM

Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.

Tez doesn't use the low level api, it use AMRMClientAsync which has shaded AllocateResponse.

https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/async/impl/AMRMClientAsyncImpl.java



Best Regard,
Jeff Zhang


From: Robert Grandl <rg...@yahoo.com>>
Reply-To: "user@tez.apache.org<ma...@tez.apache.org>" <us...@tez.apache.org>>, Robert Grandl <rg...@yahoo.com>>
Date: Monday, January 11, 2016 at 7:10 PM
To: User <us...@tez.incubator.apache.org>>, "dev@tez.incubator.apache.org<ma...@tez.incubator.apache.org>" <de...@tez.incubator.apache.org>>
Subject: Response from RM

Hi guys,

I am having a setup where I am running Tez 0.7.0 and Hadoop 2.6.0.

I am trying to propagate some additional information from RM to Tez AM. To do that, I was modifying the heartbeat response datastructure from RM to introduce an additional field. I modified AllocareResponseProto datastructure from yarn_service_protos.proto and the inherent classes in Yarn.

However, I could not find where in Tez the heartbeat response is received and hence where to get my additional information, propagated back from Yarn.

Thanks in advance,
Robert