You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by tienduc_dinh <ti...@yahoo.com> on 2009/01/11 14:50:50 UTC

General questions about Map-Reduce

hi,

I have some questions about Map-Reduce that I'm not sure, hope that you guys
can help me.

- Does Map-Reduce support parallel writing/reading ? 

I think not because I don't find anything like that in the source code. And
why the data of a input file are distributed on many data-nodes but the
parallel concept isn't used.

- What happens after the Map-Reduce operation ?

I got the first example on
http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+WordCount+v1.0

So after the Map-Reduce operation we got

< Bye, 1>
< Goodbye, 1>
< Hadoop, 2>
< Hello, 2>
< World, 2> 

But how can I know that the input file will be written/read correctly on the
file system ?

Any help will be appreciated, thanks.

Tien Duc Dinh

-- 
View this message in context: http://www.nabble.com/General-questions-about-Map-Reduce-tp21399361p21399361.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: General questions about Map-Reduce

Posted by Owen O'Malley <om...@apache.org>.
On Jan 11, 2009, at 7:05 PM, tienduc_dinh wrote:

> Is there any article which describes it ?

Please read the map/reduce tutorial:

http://hadoop.apache.org/core/docs/current/mapred_tutorial.html

-- Owen

Re: General questions about Map-Reduce

Posted by tienduc_dinh <ti...@yahoo.com>.
I got it ...

Thanks to all

Cheers,
Duc
-- 
View this message in context: http://www.nabble.com/General-questions-about-Map-Reduce-tp21399361p21461628.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: General questions about Map-Reduce

Posted by Stuart White <st...@gmail.com>.
> On Sun, Jan 11, 2009 at 9:05 PM, tienduc_dinh <ti...@yahoo.com>wrote:
>
> Is there any article which describes it ?
>

I'd also recommend Google's MapReduce whitepaper:

http://labs.google.com/papers/mapreduce.html

Re: General questions about Map-Reduce

Posted by "Philip (flip) Kromer" <fl...@infochimps.org>.
On Sun, Jan 11, 2009 at 9:05 PM, tienduc_dinh <ti...@yahoo.com>wrote:

> Is there any article which describes it ?
>

There's also Tom White's in-progress "Hadoop: The Definitive Guide":
http://my.safaribooksonline.com/9780596521974

flip
-- 
http://www.infochimps.org
Connected Open Free Data

Re: General questions about Map-Reduce

Posted by tienduc_dinh <ti...@yahoo.com>.
- What happens after the Map-Reduce operation ?

> The OutputFormat usually writes the output to HDFS.

Hi Owen, thanks for your help. 

Is there any article which describes it ?

Regards,
Tien Duc Dinh
-- 
View this message in context: http://www.nabble.com/General-questions-about-Map-Reduce-tp21399361p21407657.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: General questions about Map-Reduce

Posted by Owen O'Malley <om...@apache.org>.
On Jan 11, 2009, at 5:50 AM, tienduc_dinh wrote:

> - Does Map-Reduce support parallel writing/reading ?

Yes. The maps all read in parallel with each other and the reduces all  
write in parallel with each other.

> - What happens after the Map-Reduce operation ?

The OutputFormat usually writes the output to HDFS.

-- Owen