You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Shekhar Gupta <sh...@gmail.com> on 2012/08/23 03:23:14 UTC
How to get TaskId from ContainerId or ApplicationId or Request in
Hadoop 0.23??
In ResourceManager, is there any way to findout if the assigned container
is going to execute a mapping task or a reduce task? I can access objects
Container, Application and Request in ResourceManager, can I somehow get
TaskId by using any of these objects?? Please let me know a way.
Thanks.
Re: How to get TaskId from ContainerId or ApplicationId or Request in
Hadoop 0.23??
Posted by Shekhar Gupta <sh...@gmail.com>.
Thanks Vinod. Yes it's the second case where I am working with MR over yarn.
When I run a job then for a specific Machine I am trying to compute that
how much time the machine takes to execute mapping tasks and reduce tasks.
What I am doing now is that I compute time difference between container
assignment and container released steps. But I can't determine if that
container was assigned for a mapping or reduce task. I'll try to look into
ApplicationMaster if I can get this information.
On Thu, Aug 23, 2012 at 11:27 AM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Moving to yarn-dev, as that is the right place for this discussion.
>
> Can you let us know more about what you are trying to accomplish? Working
> with MapReduce over YARN or your own YARN application?
>
> If you are working with MR over YARN, you have to note that
> TaskId/TaskAttemptID/Map/Reduce tasks are MapReduce concepts. So
> ResourceManager has no idea about them. MapReduce ApplicationMaster is the
> place where you can obtain more such MR specific information.
>
> If it is the later, your question is moot.
>
> HTH,
> +Vinod
>
> On Aug 22, 2012, at 6:23 PM, Shekhar Gupta wrote:
>
> > In ResourceManager, is there any way to findout if the assigned container
> > is going to execute a mapping task or a reduce task? I can access objects
> > Container, Application and Request in ResourceManager, can I somehow get
> > TaskId by using any of these objects?? Please let me know a way.
> >
> > Thanks.
>
>
Re: How to get TaskId from ContainerId or ApplicationId or Request in Hadoop 0.23??
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Moving to yarn-dev, as that is the right place for this discussion.
Can you let us know more about what you are trying to accomplish? Working with MapReduce over YARN or your own YARN application?
If you are working with MR over YARN, you have to note that TaskId/TaskAttemptID/Map/Reduce tasks are MapReduce concepts. So ResourceManager has no idea about them. MapReduce ApplicationMaster is the place where you can obtain more such MR specific information.
If it is the later, your question is moot.
HTH,
+Vinod
On Aug 22, 2012, at 6:23 PM, Shekhar Gupta wrote:
> In ResourceManager, is there any way to findout if the assigned container
> is going to execute a mapping task or a reduce task? I can access objects
> Container, Application and Request in ResourceManager, can I somehow get
> TaskId by using any of these objects?? Please let me know a way.
>
> Thanks.
Re: How to get TaskId from ContainerId or ApplicationId or Request in Hadoop 0.23??
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Moving to yarn-dev, as that is the right place for this discussion.
Can you let us know more about what you are trying to accomplish? Working with MapReduce over YARN or your own YARN application?
If you are working with MR over YARN, you have to note that TaskId/TaskAttemptID/Map/Reduce tasks are MapReduce concepts. So ResourceManager has no idea about them. MapReduce ApplicationMaster is the place where you can obtain more such MR specific information.
If it is the later, your question is moot.
HTH,
+Vinod
On Aug 22, 2012, at 6:23 PM, Shekhar Gupta wrote:
> In ResourceManager, is there any way to findout if the assigned container
> is going to execute a mapping task or a reduce task? I can access objects
> Container, Application and Request in ResourceManager, can I somehow get
> TaskId by using any of these objects?? Please let me know a way.
>
> Thanks.
Re: How to get TaskId from ContainerId or ApplicationId or Request
in Hadoop 0.23??
Posted by Robert Evans <ev...@yahoo-inc.com>.
There really is no way. The RM also has no knowledge of map tasks vs
reduce tasks nor should it know.
--Bobby
On 8/22/12 8:23 PM, "Shekhar Gupta" <sh...@gmail.com> wrote:
>In ResourceManager, is there any way to findout if the assigned container
>is going to execute a mapping task or a reduce task? I can access objects
>Container, Application and Request in ResourceManager, can I somehow get
>TaskId by using any of these objects?? Please let me know a way.
>
>Thanks.