You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rajeshkumar J <ra...@gmail.com> on 2017/05/17 05:58:47 UTC

hbase coprocessor in mapreduce

Hi,

   whether I can call hase coprocessor in the map function of mapreduce job?

Thanks

Re: hbase coprocessor in mapreduce

Posted by Ted Yu <yu...@gmail.com>.
You can take a look at the following from Phoenix:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java

On Wed, May 17, 2017 at 1:59 AM, Rajeshkumar J <ra...@gmail.com>
wrote:

> Hi,
>
>   I want to groupby hbase data by some columns and again I want to group
> the resultant data by some columns and check the size of each group if
> greater than 1 it will be written to hdfs. Initially we are doing this in
> mapreduce but we got some issue in reducer. So we thought we can move this
> group by operation to hbase using coprocessor.
>   Hence using this coprocessor we will scan and group data by some columns
> . Then again I want to group the resultant data how to achieve this in
> coprocessor?? whether this use case fits in coprocessor??
>
> Thanks
>
> On Wed, May 17, 2017 at 1:28 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Can you clarify your use case ?
> >
> > If your mapper calls Table#coprocessorService(), the recipient may not be
> > on the same region server as the mapper.
> >
> > On Tue, May 16, 2017 at 10:58 PM, Rajeshkumar J <
> > rajeshkumarit8292@gmail.com
> > > wrote:
> >
> > > Hi,
> > >
> > >    whether I can call hase coprocessor in the map function of mapreduce
> > > job?
> > >
> > > Thanks
> > >
> >
>

Re: hbase coprocessor in mapreduce

Posted by Rajeshkumar J <ra...@gmail.com>.
Hi,

  I want to groupby hbase data by some columns and again I want to group
the resultant data by some columns and check the size of each group if
greater than 1 it will be written to hdfs. Initially we are doing this in
mapreduce but we got some issue in reducer. So we thought we can move this
group by operation to hbase using coprocessor.
  Hence using this coprocessor we will scan and group data by some columns
. Then again I want to group the resultant data how to achieve this in
coprocessor?? whether this use case fits in coprocessor??

Thanks

On Wed, May 17, 2017 at 1:28 PM, Ted Yu <yu...@gmail.com> wrote:

> Can you clarify your use case ?
>
> If your mapper calls Table#coprocessorService(), the recipient may not be
> on the same region server as the mapper.
>
> On Tue, May 16, 2017 at 10:58 PM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com
> > wrote:
>
> > Hi,
> >
> >    whether I can call hase coprocessor in the map function of mapreduce
> > job?
> >
> > Thanks
> >
>

Re: hbase coprocessor in mapreduce

Posted by Ted Yu <yu...@gmail.com>.
Can you clarify your use case ?

If your mapper calls Table#coprocessorService(), the recipient may not be
on the same region server as the mapper.

On Tue, May 16, 2017 at 10:58 PM, Rajeshkumar J <rajeshkumarit8292@gmail.com
> wrote:

> Hi,
>
>    whether I can call hase coprocessor in the map function of mapreduce
> job?
>
> Thanks
>