You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by Tomasz Guziałek <To...@HumanInference.com> on 2014/06/21 11:11:46 UTC
HBase LIMIT and OFFSET.
Hello
I need to effectively read a range of records from HBase, so I have been analyzing the code for support of LIMIT and OFFSET clauses in HBaseDataContext. It seems like some efforts already has been done; HBaseDataContext has an implementation of materializeMainSchemaTable method with setting maxRows. Do I understand it well that LIMIT is supported, but not OFFSET?
Best regards,
Tomasz
Re: HBase LIMIT and OFFSET.
Posted by Kasper Sørensen <i....@gmail.com>.
Hmm I think it looks like it would only work if you know the previous page.
And LIMIT/OFFSET goes beyond paging I guess, and at the point where we
evaluate the query, we don't know anything about any previous pages.
I think it's bad luck on this one ... Doesn't seem like HBase supports
something equivalent to OFFSET as defined in SQL.
2014-06-23 9:42 GMT+02:00 Tomasz Guziałek <
Tomasz.Guzialek@humaninference.com>:
> There is a possibility to set a start and end row to a Scan, but we need
> to know the value of the starting row, not its order as specified in OFFSET
> clause. However, I found this response on StackOverflow:
> http://stackoverflow.com/questions/23452700/hbase-scan-with-offset .
> Looks a little like a workaround, need to think if we want to incorporate
> that kind of solution in MetaModel.
>
> Regards,
> Tomasz
>
> -----Original Message-----
> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
> Sent: Sunday, June 22, 2014 20:08
> To: dev@metamodel.incubator.apache.org
> Subject: Re: HBase LIMIT and OFFSET.
>
> Hi Tomasz,
>
> That's right. It basically means that you functionally also have support
> for OFFSET, but it's a naive/greedy implementation which does the
> offsetting at the client side.
>
> I am not aware of a method to push down the offset criteria to the HBase
> query (scan), but if there is a way it would be an awesome improvement to
> use it. Maybe worth a little research, maybe even asking around on the
> HBase mailing lists?
>
> Kasper
>
>
> 2014-06-21 11:11 GMT+02:00 Tomasz Guziałek <
> Tomasz.Guzialek@humaninference.com>:
>
> > Hello
> >
> > I need to effectively read a range of records from HBase, so I have
> > been analyzing the code for support of LIMIT and OFFSET clauses in
> > HBaseDataContext. It seems like some efforts already has been done;
> > HBaseDataContext has an implementation of materializeMainSchemaTable
> > method with setting maxRows. Do I understand it well that LIMIT is
> > supported, but not OFFSET?
> > Best regards,
> > Tomasz
> >
>
RE: HBase LIMIT and OFFSET.
Posted by Tomasz Guziałek <To...@HumanInference.com>.
There is a possibility to set a start and end row to a Scan, but we need to know the value of the starting row, not its order as specified in OFFSET clause. However, I found this response on StackOverflow: http://stackoverflow.com/questions/23452700/hbase-scan-with-offset . Looks a little like a workaround, need to think if we want to incorporate that kind of solution in MetaModel.
Regards,
Tomasz
-----Original Message-----
From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
Sent: Sunday, June 22, 2014 20:08
To: dev@metamodel.incubator.apache.org
Subject: Re: HBase LIMIT and OFFSET.
Hi Tomasz,
That's right. It basically means that you functionally also have support for OFFSET, but it's a naive/greedy implementation which does the offsetting at the client side.
I am not aware of a method to push down the offset criteria to the HBase query (scan), but if there is a way it would be an awesome improvement to use it. Maybe worth a little research, maybe even asking around on the HBase mailing lists?
Kasper
2014-06-21 11:11 GMT+02:00 Tomasz Guziałek <
Tomasz.Guzialek@humaninference.com>:
> Hello
>
> I need to effectively read a range of records from HBase, so I have
> been analyzing the code for support of LIMIT and OFFSET clauses in
> HBaseDataContext. It seems like some efforts already has been done;
> HBaseDataContext has an implementation of materializeMainSchemaTable
> method with setting maxRows. Do I understand it well that LIMIT is
> supported, but not OFFSET?
> Best regards,
> Tomasz
>
Re: HBase LIMIT and OFFSET.
Posted by Kasper Sørensen <i....@gmail.com>.
Hi Tomasz,
That's right. It basically means that you functionally also have support
for OFFSET, but it's a naive/greedy implementation which does the
offsetting at the client side.
I am not aware of a method to push down the offset criteria to the HBase
query (scan), but if there is a way it would be an awesome improvement to
use it. Maybe worth a little research, maybe even asking around on the
HBase mailing lists?
Kasper
2014-06-21 11:11 GMT+02:00 Tomasz Guziałek <
Tomasz.Guzialek@humaninference.com>:
> Hello
>
> I need to effectively read a range of records from HBase, so I have been
> analyzing the code for support of LIMIT and OFFSET clauses in
> HBaseDataContext. It seems like some efforts already has been done;
> HBaseDataContext has an implementation of materializeMainSchemaTable method
> with setting maxRows. Do I understand it well that LIMIT is supported, but
> not OFFSET?
> Best regards,
> Tomasz
>