You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Rajesh Jain <rj...@gmail.com> on 2013/08/20 23:58:03 UTC
Method Calls
I am looking for the various method calls (entry points) in the inter tier
communication.
When the client submits a job for map-reduce, what is the sequence of calls
started, e.g Node Manager to Resource Manager or Container to App Master.
How is the App Master Child JVM spawned etc.
Does any one have a high level call flow / sequence diagram?
Thanks,
Rajesh
[image: MapReduce NextGen Architecture]
Re: Method Calls
Posted by Rajesh Jain <rj...@gmail.com>.
Thanks. Both these blogs are awesome.
On Tue, Aug 20, 2013 at 10:51 PM, Harsh J <ha...@cloudera.com> wrote:
> For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
> found at
> http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
>
>
> On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>>
>>
>> I am looking for the various method calls (entry points) in the inter
>> tier communication.
>>
>> When the client submits a job for map-reduce, what is the sequence of
>> calls started, e.g Node Manager to Resource Manager or Container to App
>> Master. How is the App Master Child JVM spawned etc.
>>
>> Does any one have a high level call flow / sequence diagram?
>>
>> Thanks,
>> Rajesh
>>
>>
>>
>> [image: MapReduce NextGen Architecture]
>>
>
>
>
> --
> Harsh J
>
Re: Method Calls
Posted by Rajesh Jain <rj...@gmail.com>.
Thanks. Both these blogs are awesome.
On Tue, Aug 20, 2013 at 10:51 PM, Harsh J <ha...@cloudera.com> wrote:
> For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
> found at
> http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
>
>
> On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>>
>>
>> I am looking for the various method calls (entry points) in the inter
>> tier communication.
>>
>> When the client submits a job for map-reduce, what is the sequence of
>> calls started, e.g Node Manager to Resource Manager or Container to App
>> Master. How is the App Master Child JVM spawned etc.
>>
>> Does any one have a high level call flow / sequence diagram?
>>
>> Thanks,
>> Rajesh
>>
>>
>>
>> [image: MapReduce NextGen Architecture]
>>
>
>
>
> --
> Harsh J
>
Re: Method Calls
Posted by Rajesh Jain <rj...@gmail.com>.
Thanks. Both these blogs are awesome.
On Tue, Aug 20, 2013 at 10:51 PM, Harsh J <ha...@cloudera.com> wrote:
> For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
> found at
> http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
>
>
> On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>>
>>
>> I am looking for the various method calls (entry points) in the inter
>> tier communication.
>>
>> When the client submits a job for map-reduce, what is the sequence of
>> calls started, e.g Node Manager to Resource Manager or Container to App
>> Master. How is the App Master Child JVM spawned etc.
>>
>> Does any one have a high level call flow / sequence diagram?
>>
>> Thanks,
>> Rajesh
>>
>>
>>
>> [image: MapReduce NextGen Architecture]
>>
>
>
>
> --
> Harsh J
>
Re: Method Calls
Posted by Rajesh Jain <rj...@gmail.com>.
Thanks. Both these blogs are awesome.
On Tue, Aug 20, 2013 at 10:51 PM, Harsh J <ha...@cloudera.com> wrote:
> For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
> found at
> http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
>
>
> On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>>
>>
>> I am looking for the various method calls (entry points) in the inter
>> tier communication.
>>
>> When the client submits a job for map-reduce, what is the sequence of
>> calls started, e.g Node Manager to Resource Manager or Container to App
>> Master. How is the App Master Child JVM spawned etc.
>>
>> Does any one have a high level call flow / sequence diagram?
>>
>> Thanks,
>> Rajesh
>>
>>
>>
>> [image: MapReduce NextGen Architecture]
>>
>
>
>
> --
> Harsh J
>
Re: Method Calls
Posted by Harsh J <ha...@cloudera.com>.
For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
found at http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
Harsh J
Re: Method Calls
Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi,
I think you should take a look at yarn architecture blog series. I found
them useful when I started working on YARN (
http://hortonworks.com/blog/introducing-apache-hadoop-yarn/) This will give
you a very good picture about communication between different modules and
roles played by them.
Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Tue, Aug 20, 2013 at 2:58 PM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Method Calls
Posted by Harsh J <ha...@cloudera.com>.
For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
found at http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
Harsh J
Re: Method Calls
Posted by Harsh J <ha...@cloudera.com>.
For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
found at http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
Harsh J
Re: Method Calls
Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi,
I think you should take a look at yarn architecture blog series. I found
them useful when I started working on YARN (
http://hortonworks.com/blog/introducing-apache-hadoop-yarn/) This will give
you a very good picture about communication between different modules and
roles played by them.
Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Tue, Aug 20, 2013 at 2:58 PM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Method Calls
Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi,
I think you should take a look at yarn architecture blog series. I found
them useful when I started working on YARN (
http://hortonworks.com/blog/introducing-apache-hadoop-yarn/) This will give
you a very good picture about communication between different modules and
roles played by them.
Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Tue, Aug 20, 2013 at 2:58 PM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Method Calls
Posted by Harsh J <ha...@cloudera.com>.
For the MR2 YARN app, a nice diagram of all relevant protocol calls can be
found at http://blog.cloudera.com/blog/2012/02/mapreduce-2-0-in-hadoop-0-23/
On Wed, Aug 21, 2013 at 3:28 AM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
Harsh J
Re: Method Calls
Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi,
I think you should take a look at yarn architecture blog series. I found
them useful when I started working on YARN (
http://hortonworks.com/blog/introducing-apache-hadoop-yarn/) This will give
you a very good picture about communication between different modules and
roles played by them.
Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Tue, Aug 20, 2013 at 2:58 PM, Rajesh Jain <rj...@gmail.com> wrote:
>
>
> I am looking for the various method calls (entry points) in the inter tier
> communication.
>
> When the client submits a job for map-reduce, what is the sequence of
> calls started, e.g Node Manager to Resource Manager or Container to App
> Master. How is the App Master Child JVM spawned etc.
>
> Does any one have a high level call flow / sequence diagram?
>
> Thanks,
> Rajesh
>
>
>
> [image: MapReduce NextGen Architecture]
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.