You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Chris Riccomini <cr...@linkedin.com> on 2011/09/20 23:26:28 UTC

ApplicationMaster appId/appAttemptId/clusterTimestamp

Hey Guys,

Sorry to bug again, but I think it's best to get this stuff into the mail archives.

I want to confirm that I'm starting my ApplicationMaster in the "proper" way.

Right now, my client is setCommand'ing this:

"java -cp './package/*' kafka.yarn.ApplicationMaster " + streamerClass + " " + tasks + " "
        + "1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout "
        + "2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"

It seems to me that my ApplicationMaster needs an appId, appAttemptId, and clusterTimestamp in order to register with the ResourceManager. The only way that I see to do this is to pass it (via the client) as the first three params to my kafka.yarn.ApplicationMaster. I recall Arun mentioning that not all of this is required, but it seems to me (based on the API) that it is.

Should I be putting the three RM params after the kafka.yarn.ApplicationMaster param in my setCommand?

Thanks!
Chris

Re: ApplicationMaster appId/appAttemptId/clusterTimestamp

Posted by Arun C Murthy <ac...@hortonworks.com>.

On Sep 20, 2011, at 5:32 PM, Hitesh Shah wrote:

> Hi Chris
> 
> You are right. At the moment, you do need to pass all of them in to your app master for it to register. I will file a jira to simplify this to end up with something along the lines where the RM implicitly passes such information to the app master without the client explicitly setting it ( the AM will need to consume the info for registration though). 
> 

Thanks Hitesh - we can use a @appattemptid@ or some such nomenclature.

> thanks
> -- Hitesh 
> 
> 
> On Sep 20, 2011, at 2:26 PM, Chris Riccomini wrote:
> 
>> Hey Guys,
>> 
>> Sorry to bug again, but I think it's best to get this stuff into the mail archives.
>> 
>> I want to confirm that I'm starting my ApplicationMaster in the "proper" way.
>> 
>> Right now, my client is setCommand'ing this:
>> 
>> "java -cp './package/*' kafka.yarn.ApplicationMaster " + streamerClass + " " + tasks + " "
>>       + "1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout "
>>       + "2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>> 
>> It seems to me that my ApplicationMaster needs an appId, appAttemptId, and clusterTimestamp in order to register with the ResourceManager. The only way that I see to do this is to pass it (via the client) as the first three params to my kafka.yarn.ApplicationMaster. I recall Arun mentioning that not all of this is required, but it seems to me (based on the API) that it is.
>> 
>> Should I be putting the three RM params after the kafka.yarn.ApplicationMaster param in my setCommand?
>> 
>> Thanks!
>> Chris
>

Re: ApplicationMaster appId/appAttemptId/clusterTimestamp

Posted by Hitesh Shah <hi...@hortonworks.com>.

Hi Chris

You are right. At the moment, you do need to pass all of them in to your app master for it to register. I will file a jira to simplify this to end up with something along the lines where the RM implicitly passes such information to the app master without the client explicitly setting it ( the AM will need to consume the info for registration though). 

thanks
-- Hitesh 


On Sep 20, 2011, at 2:26 PM, Chris Riccomini wrote:

> Hey Guys,
> 
> Sorry to bug again, but I think it's best to get this stuff into the mail archives.
> 
> I want to confirm that I'm starting my ApplicationMaster in the "proper" way.
> 
> Right now, my client is setCommand'ing this:
> 
> "java -cp './package/*' kafka.yarn.ApplicationMaster " + streamerClass + " " + tasks + " "
>        + "1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout "
>        + "2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
> 
> It seems to me that my ApplicationMaster needs an appId, appAttemptId, and clusterTimestamp in order to register with the ResourceManager. The only way that I see to do this is to pass it (via the client) as the first three params to my kafka.yarn.ApplicationMaster. I recall Arun mentioning that not all of this is required, but it seems to me (based on the API) that it is.
> 
> Should I be putting the three RM params after the kafka.yarn.ApplicationMaster param in my setCommand?
> 
> Thanks!
> Chris