Hadoop node's locality

[Renato Marroquín Mogrovejo <re...@gmail.com>]

Hi everyone,
Does anybody know if Hadoop can take scheduling decisions using nodes positions
within network topology ? Or if there is any work being done on this?
Thanks in advanced.


Renato M.

Session analysis using Hadoop

[Oded Rotem <od...@gmail.com>]

Hi,

Does anyone have any experience using Hadoop to do sessionizing / session
analysis? I know eBay has something called MQL (Mobius Query Language) for
this, but they have not (yet?...) made it open source.

Any useful pointers here would be appreciated.

Cheers,
Oded

Re: Hadoop node's locality

[Ryan Rawson <ry...@gmail.com>]

Hi,

The map-reduce scheduling does take block placement into account and
attempts to schedule map tasks accordingly.  The system is flexible,
and HBase uses it to put maps colocated with the regionservers they
read from.

-ryan

On Mon, Aug 9, 2010 at 2:32 PM, Renato Marroquín Mogrovejo
<re...@gmail.com> wrote:
> Hi, thanks for answering, but I did find the answer out. I knew Hadoop did
> it, but I needed a reference.
> I found it in "Data-intensive Text Processing with MapReduce" in the page
> 25.
> I quoted here:
>
> Data/code co-location: . . . "An important optimization here is to prefer
> nodes that are on the same rack in the datacenter as the node holding the
> relevant data block, since inter-rack bandwidth is significantly less than
> intra-rack bandwidth."
>
> Thanks again.
>
>
> Renato M.
>
>
> 2010/8/9 vaibhav negi <ss...@gmail.com>
>
>> Hi ,
>>
>> I am also a nebie to hadoop. But i think u should read about "rack aware"
>> concept.
>> It may solve your query.
>>
>> Vaibhav Negi
>>
>>
>> On Sun, Aug 8, 2010 at 12:10 AM, Renato Marroquín Mogrovejo <
>> renatoj.marroquin@gmail.com> wrote:
>>
>> > Hi everyone,
>> > Does anybody know if Hadoop can take scheduling decisions using nodes
>> > positions
>> > within network topology ? Or if there is any work being done on this?
>> > Thanks in advanced.
>> >
>> >
>> > Renato M.
>> >
>>
>

Re: Hadoop node's locality

[Renato Marroquín Mogrovejo <re...@gmail.com>]

Hi, thanks for answering, but I did find the answer out. I knew Hadoop did
it, but I needed a reference.
I found it in "Data-intensive Text Processing with MapReduce" in the page
25.
I quoted here:

Data/code co-location: . . . "An important optimization here is to prefer
nodes that are on the same rack in the datacenter as the node holding the
relevant data block, since inter-rack bandwidth is significantly less than
intra-rack bandwidth."

Thanks again.


Renato M.


2010/8/9 vaibhav negi <ss...@gmail.com>

> Hi ,
>
> I am also a nebie to hadoop. But i think u should read about "rack aware"
> concept.
> It may solve your query.
>
> Vaibhav Negi
>
>
> On Sun, Aug 8, 2010 at 12:10 AM, Renato Marroquín Mogrovejo <
> renatoj.marroquin@gmail.com> wrote:
>
> > Hi everyone,
> > Does anybody know if Hadoop can take scheduling decisions using nodes
> > positions
> > within network topology ? Or if there is any work being done on this?
> > Thanks in advanced.
> >
> >
> > Renato M.
> >
>

Re: Hadoop node's locality

[vaibhav negi <ss...@gmail.com>]

Hi ,

I am also a nebie to hadoop. But i think u should read about "rack aware"
concept.
It may solve your query.

Vaibhav Negi


On Sun, Aug 8, 2010 at 12:10 AM, Renato Marroquín Mogrovejo <
renatoj.marroquin@gmail.com> wrote:

> Hi everyone,
> Does anybody know if Hadoop can take scheduling decisions using nodes
> positions
> within network topology ? Or if there is any work being done on this?
> Thanks in advanced.
>
>
> Renato M.
>