You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by hm...@umbc.edu on 2010/06/10 13:47:59 UTC

Re: Is it possible ....!!! COOL!

Hey,

This is a really neat idea.... if anyone has a way to do this, could you
share?
I'll bet this could be very interesting! Thanks...

Best,
HAL


> Hi,
>     I wanted to ask if it is possible to intercept every communication
> that
> takes place between hadoop's map reduce task i.e between JobTracker and
> TaskTracker and make it pass through my own communication library.
> So, if JobTracker and TaskTracker talk through http or rpc, i would like
> to
> intercept the call and let it pass through my communication library. If it
> is possible can anyone tell me that which set of classes i need to look at
> hadoop's distribution.
>
> Similarly, for the hdfs, is it possible to let all the communication that
> is
> happening between namenode and datanode to pass through my communication
> library.
>
> Reason for doing that is that i want all the communication to happen
> through
> a communication library that resolves every communication problem that we
> can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By
> using that library all the headache of communication will be gone. So, we
> will be able to use hadoop quite easily and there will be no communication
> problems.
>
> Thats my master's project. So, i want to know how to start and where to
> look
> for.
>
> I would really appreciate a reply.
>
> Regards,
> Ahmad Shahzad
>

Re: Is it possible ....!!! COOL!

Posted by Ryan Smith <ry...@gmail.com>.
Sounds like it could be a SPOF.

On Thu, Jun 10, 2010 at 7:47 AM, <hm...@umbc.edu> wrote:

> Hey,
>
> This is a really neat idea.... if anyone has a way to do this, could you
> share?
> I'll bet this could be very interesting! Thanks...
>
> Best,
> HAL
>
>
> > Hi,
> >     I wanted to ask if it is possible to intercept every communication
> > that
> > takes place between hadoop's map reduce task i.e between JobTracker and
> > TaskTracker and make it pass through my own communication library.
> > So, if JobTracker and TaskTracker talk through http or rpc, i would like
> > to
> > intercept the call and let it pass through my communication library. If
> it
> > is possible can anyone tell me that which set of classes i need to look
> at
> > hadoop's distribution.
> >
> > Similarly, for the hdfs, is it possible to let all the communication that
> > is
> > happening between namenode and datanode to pass through my communication
> > library.
> >
> > Reason for doing that is that i want all the communication to happen
> > through
> > a communication library that resolves every communication problem that we
> > can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By
> > using that library all the headache of communication will be gone. So, we
> > will be able to use hadoop quite easily and there will be no
> communication
> > problems.
> >
> > Thats my master's project. So, i want to know how to start and where to
> > look
> > for.
> >
> > I would really appreciate a reply.
> >
> > Regards,
> > Ahmad Shahzad
> >
>
>
>