You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by neo21 zerro <ne...@yahoo.com> on 2012/01/27 11:58:15 UTC
Question about mapReduce.
Hello,
I'm really new to Hadoop and I was wondering if the MAP reduce programming model from Hadoop is a good choice only for processing large amount of data, from a file, database or a queue? Thanks!
Re: Question about mapReduce.
Posted by Ronald Petty <ro...@gmail.com>.
neo21 zerro,
*
*
Hadoop may or may not be able to help you based on your problem
specification. Based on your statement "processes big data files" it
should be able to help. Relating to that, I am unclear why HDFS is an
issue? As for queues and databases, can you describe what you have in mind?
I think someone can provide you more concrete information (even about
Hadoop proper) with some more details on your specific concerns.
Regards.
Ron
*
*
On Sat, Jan 28, 2012 at 12:01 PM, neo21 zerro <ne...@yahoo.com> wrote:
> So basically if I don't have the data into the HDFS sistem, the map
> reduce from HADOOP will not help me ?
> Because I need to build a tool that processes big data files, a lot of
> messages from queues or databases,
> and I thought that by using the map reduce from HADOOP my life would be
> easier :)
>
> Thanks for the answers
>
> ------------------------------
> *From:* real great.. <gr...@gmail.com>
> *To:* mapreduce-user@hadoop.apache.org
> *Sent:* Friday, January 27, 2012 5:15 PM
> *Subject:* Re: Question about mapReduce.
>
>
> I think his question was more different.Not sure though.
> On Fri, Jan 27, 2012 at 5:21 PM, Ashwanth Kumar <
> ashwanthkumar@googlemail.com> wrote:
>
> Sorry Harsh, it was quite some time since I followed Sqoop. Thanks for the
> update.
>
> - Ashwanth
>
> On Fri, Jan 27, 2012 at 5:07 PM, Harsh J <ha...@cloudera.com> wrote:
>
> Small correction to Ashwanth's post - Sqoop is now an Apache Incubator
> project residing at http://incubator.apache.org/sqoop with a community
> of its own.
>
> On Fri, Jan 27, 2012 at 4:56 PM, Ashwanth Kumar
> <as...@googlemail.com> wrote:
> > Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
> > Cloudera) imports data from SQL databases to HDFS for processing.
> Everything
> > that is processed in Hadoop is only from HDFS. So if you can put
> anything in
> > a HDFS, you can process anything.
> >
> > - Ashwanth
> >
> >
> > On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com>
> wrote:
> >>
> >>
> >> Hello,
> >>
> >>
> >> I'm really new to Hadoop and I was wondering if the MAP reduce
> >> programming model from Hadoop is a good choice only for processing
> large
> >> amount of data, from a file, database or a queue? Thanks!
> >
> >
>
>
>
> --
> Harsh J
> Customer Ops. Engineer, Cloudera
>
>
>
>
>
> --
> Regards,
> R.V.
>
>
>
Re: Question about mapReduce.
Posted by neo21 zerro <ne...@yahoo.com>.
So basically if I don't have the data into the HDFS sistem, the map reduce from HADOOP will not help me ?
Because I need to build a tool that processes big data files, a lot of messages from queues or databases,
and I thought that by using the map reduce from HADOOP my life would be easier :)
Thanks for the answers
________________________________
From: real great.. <gr...@gmail.com>
To: mapreduce-user@hadoop.apache.org
Sent: Friday, January 27, 2012 5:15 PM
Subject: Re: Question about mapReduce.
I think his question was more different.Not sure though.
On Fri, Jan 27, 2012 at 5:21 PM, Ashwanth Kumar <as...@googlemail.com> wrote:
Sorry Harsh, it was quite some time since I followed Sqoop. Thanks for the update.
>
>- Ashwanth
>
>
>On Fri, Jan 27, 2012 at 5:07 PM, Harsh J <ha...@cloudera.com> wrote:
>
>Small correction to Ashwanth's post - Sqoop is now an Apache Incubator
>>project residing at http://incubator.apache.org/sqoop with a community
>>of its own.
>>
>>
>>On Fri, Jan 27, 2012 at 4:56 PM, Ashwanth Kumar
>><as...@googlemail.com> wrote:
>>> Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
>>> Cloudera) imports data from SQL databases to HDFS for processing. Everything
>>> that is processed in Hadoop is only from HDFS. So if you can put anything in
>>> a HDFS, you can process anything.
>>>
>>> - Ashwanth
>>>
>>>
>>> On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com> wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>>
>>>> I'm really new to Hadoop and I was wondering if the MAP reduce
>>>> programming model from Hadoop is a good choice only for processing large
>>>> amount of data, from a file, database or a queue? Thanks!
>>>
>>>
>>
>>
>>
>>--
>>Harsh J
>>Customer Ops. Engineer, Cloudera
>>
>
--
Regards,
R.V.
Re: Question about mapReduce.
Posted by "real great.." <gr...@gmail.com>.
I think his question was more different.Not sure though.
On Fri, Jan 27, 2012 at 5:21 PM, Ashwanth Kumar <
ashwanthkumar@googlemail.com> wrote:
> Sorry Harsh, it was quite some time since I followed Sqoop. Thanks for the
> update.
>
> - Ashwanth
>
> On Fri, Jan 27, 2012 at 5:07 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Small correction to Ashwanth's post - Sqoop is now an Apache Incubator
>> project residing at http://incubator.apache.org/sqoop with a community
>> of its own.
>>
>> On Fri, Jan 27, 2012 at 4:56 PM, Ashwanth Kumar
>> <as...@googlemail.com> wrote:
>> > Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
>> > Cloudera) imports data from SQL databases to HDFS for processing.
>> Everything
>> > that is processed in Hadoop is only from HDFS. So if you can put
>> anything in
>> > a HDFS, you can process anything.
>> >
>> > - Ashwanth
>> >
>> >
>> > On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com>
>> wrote:
>> >>
>> >>
>> >> Hello,
>> >>
>> >>
>> >> I'm really new to Hadoop and I was wondering if the MAP reduce
>> >> programming model from Hadoop is a good choice only for processing
>> large
>> >> amount of data, from a file, database or a queue? Thanks!
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>> Customer Ops. Engineer, Cloudera
>>
>
>
--
Regards,
R.V.
Re: Question about mapReduce.
Posted by Ashwanth Kumar <as...@googlemail.com>.
Sorry Harsh, it was quite some time since I followed Sqoop. Thanks for the
update.
- Ashwanth
On Fri, Jan 27, 2012 at 5:07 PM, Harsh J <ha...@cloudera.com> wrote:
> Small correction to Ashwanth's post - Sqoop is now an Apache Incubator
> project residing at http://incubator.apache.org/sqoop with a community
> of its own.
>
> On Fri, Jan 27, 2012 at 4:56 PM, Ashwanth Kumar
> <as...@googlemail.com> wrote:
> > Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
> > Cloudera) imports data from SQL databases to HDFS for processing.
> Everything
> > that is processed in Hadoop is only from HDFS. So if you can put
> anything in
> > a HDFS, you can process anything.
> >
> > - Ashwanth
> >
> >
> > On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com>
> wrote:
> >>
> >>
> >> Hello,
> >>
> >>
> >> I'm really new to Hadoop and I was wondering if the MAP reduce
> >> programming model from Hadoop is a good choice only for processing
> large
> >> amount of data, from a file, database or a queue? Thanks!
> >
> >
>
>
>
> --
> Harsh J
> Customer Ops. Engineer, Cloudera
>
Re: Question about mapReduce.
Posted by Harsh J <ha...@cloudera.com>.
Small correction to Ashwanth's post - Sqoop is now an Apache Incubator
project residing at http://incubator.apache.org/sqoop with a community
of its own.
On Fri, Jan 27, 2012 at 4:56 PM, Ashwanth Kumar
<as...@googlemail.com> wrote:
> Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
> Cloudera) imports data from SQL databases to HDFS for processing. Everything
> that is processed in Hadoop is only from HDFS. So if you can put anything in
> a HDFS, you can process anything.
>
> - Ashwanth
>
>
> On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com> wrote:
>>
>>
>> Hello,
>>
>>
>> I'm really new to Hadoop and I was wondering if the MAP reduce
>> programming model from Hadoop is a good choice only for processing large
>> amount of data, from a file, database or a queue? Thanks!
>
>
--
Harsh J
Customer Ops. Engineer, Cloudera
Re: Question about mapReduce.
Posted by Ashwanth Kumar <as...@googlemail.com>.
Hadoop is very good at processing data from HDFS. Tools like Sqoop (from
Cloudera) imports data from SQL databases to HDFS for processing.
Everything that is processed in Hadoop is only from HDFS. So if you can put
anything in a HDFS, you can process anything.
- Ashwanth
On Fri, Jan 27, 2012 at 4:28 PM, neo21 zerro <ne...@yahoo.com> wrote:
>
> Hello,
>
>
> I'm really new to Hadoop and I was wondering if the MAP reduce
> programming model from Hadoop is a good choice only for processing large
> amount of data, from a file, database or a queue? Thanks!
>