You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Hua Su <hu...@gmail.com> on 2010/03/06 16:59:24 UTC
Does the scanner interface have a "previous" method?
Hi,
I've read the API doc of ResultScanner and Scanner. Both of them support to
get "next" data record in the table.
My question is, does there exist any scanner interface that supports to get
"previous" data record in the table?
Thanks!
- Hua
Re: Does the scanner interface have a "previous" method?
Posted by Ryan Rawson <ry...@gmail.com>.
Hi,
The (in)ability to do previous scans is baked in at a very low level.
Allowing for a reverse-scan might add a significant amount of storage
requirements (10-20% is non-trival on multi-TB data sets). Just like
adding in the second link for a doubly linked list isn't cheap,
neither would be reverse scanning.
-ryan
On Sun, Mar 7, 2010 at 6:36 PM, Hua Su <hu...@gmail.com> wrote:
> On Sun, Mar 7, 2010 at 11:20 PM, Michael Segel <mi...@hotmail.com>wrote:
>
>>
>> What you're attempting to recreate is a 'scroll cursor'.
>> On very large data sets, this isn't trivial to do.
>>
>> I think if you look at the use cases, you'll think of some other way of
>> achieving your specific goals.
>>
>> Just my opinion...
>>
>
> Thank. I just do some experiments and the data set is not very large.
>
>
>>
>> -Mike
>>
>>
>> > Date: Sat, 6 Mar 2010 19:38:19 -0800
>> > Subject: Re: Does the scanner interface have a "previous" method?
>> > From: erikholstad@gmail.com
>> > To: hbase-user@hadoop.apache.org
>> >
>> > Hey Hua!
>> > I agree that this would be a nice to have feature for cases when you want
>> to
>> > fetch data in
>> > ascending and descending order. You would need to store any extra indices
>> > just open up
>> > a different scanner.
>> > The best way to do this now though is to store it in a different index.
>> >
>> > Good luck!
>> > Erik
>>
>> _________________________________________________________________
>> Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
>> http://clk.atdmt.com/GBL/go/201469226/direct/01/
>>
>
Re: Does the scanner interface have a "previous" method?
Posted by Hua Su <hu...@gmail.com>.
On Sun, Mar 7, 2010 at 11:20 PM, Michael Segel <mi...@hotmail.com>wrote:
>
> What you're attempting to recreate is a 'scroll cursor'.
> On very large data sets, this isn't trivial to do.
>
> I think if you look at the use cases, you'll think of some other way of
> achieving your specific goals.
>
> Just my opinion...
>
Thank. I just do some experiments and the data set is not very large.
>
> -Mike
>
>
> > Date: Sat, 6 Mar 2010 19:38:19 -0800
> > Subject: Re: Does the scanner interface have a "previous" method?
> > From: erikholstad@gmail.com
> > To: hbase-user@hadoop.apache.org
> >
> > Hey Hua!
> > I agree that this would be a nice to have feature for cases when you want
> to
> > fetch data in
> > ascending and descending order. You would need to store any extra indices
> > just open up
> > a different scanner.
> > The best way to do this now though is to store it in a different index.
> >
> > Good luck!
> > Erik
>
> _________________________________________________________________
> Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
> http://clk.atdmt.com/GBL/go/201469226/direct/01/
>
RE: Does the scanner interface have a "previous" method?
Posted by Michael Segel <mi...@hotmail.com>.
What you're attempting to recreate is a 'scroll cursor'.
On very large data sets, this isn't trivial to do.
I think if you look at the use cases, you'll think of some other way of achieving your specific goals.
Just my opinion...
-Mike
> Date: Sat, 6 Mar 2010 19:38:19 -0800
> Subject: Re: Does the scanner interface have a "previous" method?
> From: erikholstad@gmail.com
> To: hbase-user@hadoop.apache.org
>
> Hey Hua!
> I agree that this would be a nice to have feature for cases when you want to
> fetch data in
> ascending and descending order. You would need to store any extra indices
> just open up
> a different scanner.
> The best way to do this now though is to store it in a different index.
>
> Good luck!
> Erik
_________________________________________________________________
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
http://clk.atdmt.com/GBL/go/201469226/direct/01/
Re: Does the scanner interface have a "previous" method?
Posted by Erik Holstad <er...@gmail.com>.
Hey Hua!
I agree that this would be a nice to have feature for cases when you want to
fetch data in
ascending and descending order. You would need to store any extra indices
just open up
a different scanner.
The best way to do this now though is to store it in a different index.
Good luck!
Erik
Re: Does the scanner interface have a "previous" method?
Posted by Hua Su <hu...@gmail.com>.
see inline.
Thanks,
- Hua
On Sun, Mar 7, 2010 at 6:17 AM, Stack <st...@duboce.net> wrote:
> On Sat, Mar 6, 2010 at 7:59 AM, Hua Su <hu...@gmail.com> wrote:
> > My question is, does there exist any scanner interface that supports to
> get
> > "previous" data record in the table?
> >
> No.
>
> Previous would require seeking our backward in storefiles. We have
> something that sort of does this already for the catalog tables and
> its an expensive call.
>
> Do you really need a previous? You want to walk your table in reverse?
>
If it's too expensive to implement a native 'get previous' function, I can
use some workaround to achieve the similar function.
>
> St.Ack
>
Re: Does the scanner interface have a "previous" method?
Posted by Stack <st...@duboce.net>.
On Sat, Mar 6, 2010 at 7:59 AM, Hua Su <hu...@gmail.com> wrote:
> My question is, does there exist any scanner interface that supports to get
> "previous" data record in the table?
>
No.
Previous would require seeking our backward in storefiles. We have
something that sort of does this already for the catalog tables and
its an expensive call.
Do you really need a previous? You want to walk your table in reverse?
St.Ack