Writting to HBASE in map phase

[Pablo Musa <pa...@psafe.com>]

Hey guys,
I need to update a field in an HBASE table and I want to do a mapred job for that.
I can do it using both map and red phase. However, it does not make any sense
to me, since map will pass the "Result" it is receiving to the reduce phase.

I also read in the hbase mailing list someone saying that you do not need the reduce,
which I agree. However I do not know, and did not find any hints on configuring
the map phase to write into HBASE.

I could use the HBASE API and write into the table, but I am not sure it is a good idea.

Thanks,
Pablo

RE: Writting to HBASE in map phase

[Pablo Musa <pa...@psafe.com>]

Great,
Thank you very much!

Abs,
Pablo

-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com] 
Sent: terça-feira, 10 de julho de 2012 17:28
To: mapreduce-user@hadoop.apache.org
Subject: Re: Writting to HBASE in map phase

Hi Pablo,

Just set your number of reduces to zero, and your TableOutputFormat will automatically kick in at the Map stage, i.e., just add the below to your driver before you submit the job. You need no other changes.

job.setNumReduceTasks(0);

On Wed, Jul 11, 2012 at 1:53 AM, Pablo Musa <pa...@psafe.com> wrote:
> Hey guys,
>
> I need to update a field in an HBASE table and I want to do a mapred 
> job for that.
>
> I can do it using both map and red phase. However, it does not make 
> any sense
>
> to me, since map will pass the "Result" it is receiving to the reduce phase.
>
>
>
> I also read in the hbase mailing list someone saying that you do not 
> need the reduce,
>
> which I agree. However I do not know, and did not find any hints on 
> configuring
>
> the map phase to write into HBASE.
>
>
>
> I could use the HBASE API and write into the table, but I am not sure 
> it is a good idea.
>
>
>
> Thanks,
>
> Pablo



--
Harsh J

Re: Writting to HBASE in map phase

[Harsh J <ha...@cloudera.com>]

Hi Pablo,

Just set your number of reduces to zero, and your TableOutputFormat
will automatically kick in at the Map stage, i.e., just add the below
to your driver before you submit the job. You need no other changes.

job.setNumReduceTasks(0);

On Wed, Jul 11, 2012 at 1:53 AM, Pablo Musa <pa...@psafe.com> wrote:
> Hey guys,
>
> I need to update a field in an HBASE table and I want to do a mapred job for
> that.
>
> I can do it using both map and red phase. However, it does not make any
> sense
>
> to me, since map will pass the “Result” it is receiving to the reduce phase.
>
>
>
> I also read in the hbase mailing list someone saying that you do not need
> the reduce,
>
> which I agree. However I do not know, and did not find any hints on
> configuring
>
> the map phase to write into HBASE.
>
>
>
> I could use the HBASE API and write into the table, but I am not sure it is
> a good idea.
>
>
>
> Thanks,
>
> Pablo



-- 
Harsh J