Use of RPC Server and Client...

[Ahmad Shahzad <as...@gmail.com>]

Hi ALL,
            Can anyone tell me the purpose for RPC server and RPC client
that is used in hadoop. As far as logs tell, they are used by JobTracker and
TaskTracker, as well as namenode and datanode. Can anyone tell me that for
what purpose RPC server and client are used by each of them.

Ahmad

Re: Use of RPC Server and Client...

[Jitendra Nath Pandey <ji...@yahoo-inc.com>]

Namenode and jobtracker also startup an http server which is jetty. The http interface is used to look at namenode health, status of datanodes, to browse the file system, or in case of job tracker to view job status, task tracker status etc.

JT and TT also use the same RPC mechanism to talk to namenode using the file system interface.

On 8/12/10 6:59 PM, "Ahmad Shahzad" <as...@gmail.com> wrote:

Hi Momina,
                   Thanks for replying. I wanted to know that JT and TT
interact with each other through this RPC Server and Client model, What
about the http Server Jetty.... what is it used for ??

Secondly, i would like to ask a related question, how does JT and TT
communicates with namenode and datanode?? Do they use the same RPC server
client mechanism or is there something else used for communication between
them.

Ahmad

Re: Use of RPC Server and Client...

[Ahmad Shahzad <as...@gmail.com>]

Hi Momina,
                   Thanks for replying. I wanted to know that JT and TT
interact with each other through this RPC Server and Client model, What
about the http Server Jetty.... what is it used for ??

Secondly, i would like to ask a related question, how does JT and TT
communicates with namenode and datanode?? Do they use the same RPC server
client mechanism or is there something else used for communication between
them.

Ahmad

Re: Use of RPC Server and Client...

[momina khan <mo...@gmail.com>]

all communication regarding health of task tracker and tasks assigned to TT
from JT, info on failed tasks, completed all is communicated through
heartbeat ... what other communication are you talking about? hadoop uses
the regular heartbeat and all other info piggy backs on it ... from TT to JT
on heatbeat and in the reverse on heartbeatresponse!

cheers
momina

On Fri, Aug 13, 2010 at 6:24 AM, Ahmad Shahzad <as...@gmail.com> wrote:

> Hi Momina,
>                   Is that all that RPC Server and Client is used for ???
> What about the rest of the communication that happens between Jobtracker
> and
> Tasktracker?? how the rest of the communication is done??
>
> Ahmad
>

Re: Use of RPC Server and Client...

[Ahmad Shahzad <as...@gmail.com>]

Hi Momina,
                   Is that all that RPC Server and Client is used for ???
What about the rest of the communication that happens between Jobtracker and
Tasktracker?? how the rest of the communication is done??

Ahmad

Re: Use of RPC Server and Client...

[momina khan <mo...@gmail.com>]

hi

the periodic heartbeat call that task trackers make to the jobtracker is
thru RPC ....same for namenode and data nodes.

momina

On Fri, Aug 13, 2010 at 5:39 AM, Ahmad Shahzad <as...@gmail.com> wrote:

> Hi ALL,
>            Can anyone tell me the purpose for RPC server and RPC client
> that is used in hadoop. As far as logs tell, they are used by JobTracker
> and
> TaskTracker, as well as namenode and datanode. Can anyone tell me that for
> what purpose RPC server and client are used by each of them.
>
> Ahmad
>