You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by GuoWei <we...@wbkit.com> on 2013/04/19 11:49:53 UTC
Coreprocessor always scan the whole table.
Hello,
We use HBase core processor endpoint to process realtime data. But when I use coreprocessorExec method to scan table and pass startRow and endRow. It always scan all table instead of the result between the startRow and endRow.
my code.
results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
public Hashtable<String,Hashtable> call(IEndPoint_SA instance)throws IOException{
Hashtable<String,Hashtable> s = null;
try {
s=instance.GetList();
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return s;
}
});
Best Regards / 商祺
郭伟 Guo Wei
-----------------------------------------------------
Re: Coreprocessor always scan the whole table.
Posted by ramkrishna vasudevan <ra...@gmail.com>.
HBASE-6870 deals with it. It is not yet committed. We can review the patch
and take it to closure.
Regards
Ram
On Fri, Apr 19, 2013 at 3:19 PM, GuoWei <we...@wbkit.com> wrote:
> Hello,
>
> We use HBase core processor endpoint to process realtime data. But when I
> use coreprocessorExec method to scan table and pass startRow and endRow. It
> always scan all table instead of the result between the startRow and endRow.
>
> my code.
>
> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
> new
> Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
> public Hashtable<String,Hashtable> call(IEndPoint_SA
> instance)throws IOException{
> Hashtable<String,Hashtable> s = null;
> try {
> s=instance.GetList();
> } catch (ParseException e) {
> // TODO Auto-generated catch block
> e.printStackTrace();
> }
> return s;
> }
> });
>
>
>
> Best Regards / 商祺
> 郭伟 Guo Wei
> -----------------------------------------------------
>
Re: Coreprocessor always scan the whole table.
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Then https://issues.apache.org/jira/browse/HBASE-6870 is most probably
impacting you.
Take a look at the link. It's not yet fixed but it's coming. You might want
to upgrade to a release which will include this fix.
JM
2013/4/19 GuoWei <we...@wbkit.com>
>
> We use base 0.94.1 in our production environment.
>
>
> Best Regards / 商祺
> 郭伟 Guo Wei
>
> 在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
>
> > Which hbase version are you using ?
> >
> > Thanks
> >
> > On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
> >
> >> Hello,
> >>
> >> We use HBase core processor endpoint to process realtime data. But
> when I use coreprocessorExec method to scan table and pass startRow and
> endRow. It always scan all table instead of the result between the startRow
> and endRow.
> >>
> >> my code.
> >>
> >> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
> >> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
> >> public Hashtable<String,Hashtable> call(IEndPoint_SA
> instance)throws IOException{
> >> Hashtable<String,Hashtable> s = null;
> >> try {
> >> s=instance.GetList();
> >> } catch (ParseException e) {
> >> // TODO Auto-generated catch block
> >> e.printStackTrace();
> >> }
> >> return s;
> >> }
> >> });
> >>
> >>
> >>
> >> Best Regards / 商祺
> >> 郭伟 Guo Wei
> >> -----------------------------------------------------
> >
>
>
Re: Coreprocessor always scan the whole table.
Posted by GuoWei <we...@wbkit.com>.
Thanks a lot.
Best Regards / 商祺
郭伟 Guo Wei
> Please upgrade to 0.94.6.1 which is more stable.
>
> Cheers
>
> On Apr 19, 2013, at 4:58 AM, GuoWei <we...@wbkit.com> wrote:
>
>>
>> We use base 0.94.1 in our production environment.
>>
>>
>> Best Regards / 商祺
>> 郭伟 Guo Wei
>>
>> 在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
>>
>>> Which hbase version are you using ?
>>>
>>> Thanks
>>>
>>> On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> We use HBase core processor endpoint to process realtime data. But when I use coreprocessorExec method to scan table and pass startRow and endRow. It always scan all table instead of the result between the startRow and endRow.
>>>>
>>>> my code.
>>>>
>>>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
>>>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
>>>> public Hashtable<String,Hashtable> call(IEndPoint_SA instance)throws IOException{
>>>> Hashtable<String,Hashtable> s = null;
>>>> try {
>>>> s=instance.GetList();
>>>> } catch (ParseException e) {
>>>> // TODO Auto-generated catch block
>>>> e.printStackTrace();
>>>> }
>>>> return s;
>>>> }
>>>> });
>>>>
>>>>
>>>>
>>>> Best Regards / 商祺
>>>> 郭伟 Guo Wei
>>>> -----------------------------------------------------
>>>
>>
>
Re: Coreprocessor always scan the whole table.
Posted by GuoWei <we...@wbkit.com>.
Yes, I just want to limit the scan with the table by the table row key.
And I don't pass the startRow and endRow in GetList method.
I just pass the startRow and endRow to coprocessorExec method.
My Code is as below:
>>>>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
>>>>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
>>>>> public Hashtable<String,Hashtable> call(IEndPoint_SA
>> instance)throws IOException{
>>>>> Hashtable<String,Hashtable> s = null;
>>>>> try {
>>>>> s=instance.GetList();
>>>>> } catch (ParseException e) {
>>>>> // TODO Auto-generated catch block
>>>>> e.printStackTrace();
>>>>> }
>>>>> return s;
>>>>> }
>>>>> });
Thanks a lot.
Best Regards / 商祺
郭伟 Guo Wei
在 2013-4-20，上午12:17，Gary Helmling <gh...@gmail.com> 写道：
> As others mention HBASE-6870 is about coprocessorExec() always scanning the
> full .META. table to determine region locations. Is this what you mean or
> are you talking about your coprocessor always scanning your full user table?
>
> If you want to limit the scan within regions in your user table, you'll
> need to pass startRow and endRow as parameters to your instance.GetList()
> method. Then when you create the region scanner in your coprocessor code,
> you'll need to set the start and end row yourself in order to limit the
> rows scanned.
>
>
> On Fri, Apr 19, 2013 at 5:59 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Please upgrade to 0.94.6.1 which is more stable.
>>
>> Cheers
>>
>> On Apr 19, 2013, at 4:58 AM, GuoWei <we...@wbkit.com> wrote:
>>
>>>
>>> We use base 0.94.1 in our production environment.
>>>
>>>
>>> Best Regards / 商祺
>>> 郭伟 Guo Wei
>>>
>>> 在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
>>>
>>>> Which hbase version are you using ?
>>>>
>>>> Thanks
>>>>
>>>> On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> We use HBase core processor endpoint to process realtime data. But
>> when I use coreprocessorExec method to scan table and pass startRow and
>> endRow. It always scan all table instead of the result between the startRow
>> and endRow.
>>>>>
>>>>> my code.
>>>>>
>>>>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
>>>>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
>>>>> public Hashtable<String,Hashtable> call(IEndPoint_SA
>> instance)throws IOException{
>>>>> Hashtable<String,Hashtable> s = null;
>>>>> try {
>>>>> s=instance.GetList();
>>>>> } catch (ParseException e) {
>>>>> // TODO Auto-generated catch block
>>>>> e.printStackTrace();
>>>>> }
>>>>> return s;
>>>>> }
>>>>> });
>>>>>
>>>>>
>>>>>
>>>>> Best Regards / 商祺
>>>>> 郭伟 Guo Wei
>>>>> -----------------------------------------------------
>>>>
>>>
>>
Re: Coreprocessor always scan the whole table.
Posted by Gary Helmling <gh...@gmail.com>.
As others mention HBASE-6870 is about coprocessorExec() always scanning the
full .META. table to determine region locations. Is this what you mean or
are you talking about your coprocessor always scanning your full user table?
If you want to limit the scan within regions in your user table, you'll
need to pass startRow and endRow as parameters to your instance.GetList()
method. Then when you create the region scanner in your coprocessor code,
you'll need to set the start and end row yourself in order to limit the
rows scanned.
On Fri, Apr 19, 2013 at 5:59 AM, Ted Yu <yu...@gmail.com> wrote:
> Please upgrade to 0.94.6.1 which is more stable.
>
> Cheers
>
> On Apr 19, 2013, at 4:58 AM, GuoWei <we...@wbkit.com> wrote:
>
> >
> > We use base 0.94.1 in our production environment.
> >
> >
> > Best Regards / 商祺
> > 郭伟 Guo Wei
> >
> > 在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
> >
> >> Which hbase version are you using ?
> >>
> >> Thanks
> >>
> >> On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> We use HBase core processor endpoint to process realtime data. But
> when I use coreprocessorExec method to scan table and pass startRow and
> endRow. It always scan all table instead of the result between the startRow
> and endRow.
> >>>
> >>> my code.
> >>>
> >>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
> >>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
> >>> public Hashtable<String,Hashtable> call(IEndPoint_SA
> instance)throws IOException{
> >>> Hashtable<String,Hashtable> s = null;
> >>> try {
> >>> s=instance.GetList();
> >>> } catch (ParseException e) {
> >>> // TODO Auto-generated catch block
> >>> e.printStackTrace();
> >>> }
> >>> return s;
> >>> }
> >>> });
> >>>
> >>>
> >>>
> >>> Best Regards / 商祺
> >>> 郭伟 Guo Wei
> >>> -----------------------------------------------------
> >>
> >
>
Re: Coreprocessor always scan the whole table.
Posted by Ted Yu <yu...@gmail.com>.
Please upgrade to 0.94.6.1 which is more stable.
Cheers
On Apr 19, 2013, at 4:58 AM, GuoWei <we...@wbkit.com> wrote:
>
> We use base 0.94.1 in our production environment.
>
>
> Best Regards / 商祺
> 郭伟 Guo Wei
>
> 在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
>
>> Which hbase version are you using ?
>>
>> Thanks
>>
>> On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
>>
>>> Hello,
>>>
>>> We use HBase core processor endpoint to process realtime data. But when I use coreprocessorExec method to scan table and pass startRow and endRow. It always scan all table instead of the result between the startRow and endRow.
>>>
>>> my code.
>>>
>>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
>>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
>>> public Hashtable<String,Hashtable> call(IEndPoint_SA instance)throws IOException{
>>> Hashtable<String,Hashtable> s = null;
>>> try {
>>> s=instance.GetList();
>>> } catch (ParseException e) {
>>> // TODO Auto-generated catch block
>>> e.printStackTrace();
>>> }
>>> return s;
>>> }
>>> });
>>>
>>>
>>>
>>> Best Regards / 商祺
>>> 郭伟 Guo Wei
>>> -----------------------------------------------------
>>
>
Re: Coreprocessor always scan the whole table.
Posted by GuoWei <we...@wbkit.com>.
We use base 0.94.1 in our production environment.
Best Regards / 商祺
郭伟 Guo Wei
在 2013-4-19，下午6:01，Ted Yu <yu...@gmail.com> 写道：
> Which hbase version are you using ?
>
> Thanks
>
> On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
>
>> Hello,
>>
>> We use HBase core processor endpoint to process realtime data. But when I use coreprocessorExec method to scan table and pass startRow and endRow. It always scan all table instead of the result between the startRow and endRow.
>>
>> my code.
>>
>> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
>> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
>> public Hashtable<String,Hashtable> call(IEndPoint_SA instance)throws IOException{
>> Hashtable<String,Hashtable> s = null;
>> try {
>> s=instance.GetList();
>> } catch (ParseException e) {
>> // TODO Auto-generated catch block
>> e.printStackTrace();
>> }
>> return s;
>> }
>> });
>>
>>
>>
>> Best Regards / 商祺
>> 郭伟 Guo Wei
>> -----------------------------------------------------
>
Re: Coreprocessor always scan the whole table.
Posted by Ted Yu <yu...@gmail.com>.
Which hbase version are you using ?
Thanks
On Apr 19, 2013, at 2:49 AM, GuoWei <we...@wbkit.com> wrote:
> Hello,
>
> We use HBase core processor endpoint to process realtime data. But when I use coreprocessorExec method to scan table and pass startRow and endRow. It always scan all table instead of the result between the startRow and endRow.
>
> my code.
>
> results = table.coprocessorExec(IEndPoint_SA.class, startrow, endrow,
> new Batch.Call<IEndPoint_SA,Hashtable<String,Hashtable>>() {
> public Hashtable<String,Hashtable> call(IEndPoint_SA instance)throws IOException{
> Hashtable<String,Hashtable> s = null;
> try {
> s=instance.GetList();
> } catch (ParseException e) {
> // TODO Auto-generated catch block
> e.printStackTrace();
> }
> return s;
> }
> });
>
>
>
> Best Regards / 商祺
> 郭伟 Guo Wei
> -----------------------------------------------------