You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Dave <da...@gmail.com> on 2015/03/04 13:26:55 UTC
Pagination using Java API wrong behaviour
Hi,
I'm using Hbase 0.98.4.2.2.0.0-2041-hadoop2 running on 9 nodes. My table
distributed to 12 regions and contains about 113M records.
I'm running pagination query using
/Filter pageFilter = new PageFilter(pageSize);
Scan scan = new Scan();
RegexStringComparator comp = new RegexStringComparator("._1");
RowFilter rowFilter = new RowFilter(CompareOp.EQUAL, comp);
FilterList filterList = new FilterList(Operator.MUST_PASS_ALL, pageFilter,
rowFilter);
scan.setFilter(filterList);/
My page size is 100K, on page 30 query returns 0 results, hence I get only
3M results, but when I run query using hbase shell I get 14M.
Here is hbase shell query:
/scan 'mgrs', {COLUMNS => 'mtf:fences',FILTER =>
org.apache.hadoop.hbase.filter.RowFilter.new(CompareFilter::CompareOp.valueOf('EQUAL'),RegexStringComparator.new("._1"))}/
Why my pagination Java query returns less results than hbase shell query?
Maybe I miss some configuration on client side?
thanks.
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Pagination-using-Java-API-wrong-behaviour-tp4068842.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: Pagination using Java API wrong behaviour
Posted by David <da...@gmail.com>.
I'll try to reproduce this error as unit test.
Meantime I found that number of pages always about 30 (expected 140), on
page number 11-13 Scan returns page x10 bigger than page size I asked to
return. In this "big" page Scan skip over a lot of rows.
It looks like there is some limitation on number of paged rows.
I tried to increase heap memory on client side, run my application as hadoop
- jar process, decouple PageFilter and RowFilter, open HBase connection on
each page scan, run the same scenario on another cluster.
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Pagination-using-Java-API-wrong-behaviour-tp4068842p4068963.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: Pagination using Java API wrong behaviour
Posted by Ted Yu <yu...@gmail.com>.
If you can reproduce this scenario in a unit test, I will definitely debug
it.
Cheers
On Wed, Mar 4, 2015 at 4:26 AM, Dave <da...@gmail.com> wrote:
> Hi,
> I'm using Hbase 0.98.4.2.2.0.0-2041-hadoop2 running on 9 nodes. My table
> distributed to 12 regions and contains about 113M records.
>
> I'm running pagination query using
>
> /Filter pageFilter = new PageFilter(pageSize);
> Scan scan = new Scan();
> RegexStringComparator comp = new RegexStringComparator("._1");
> RowFilter rowFilter = new RowFilter(CompareOp.EQUAL, comp);
> FilterList filterList = new FilterList(Operator.MUST_PASS_ALL, pageFilter,
> rowFilter);
> scan.setFilter(filterList);/
>
> My page size is 100K, on page 30 query returns 0 results, hence I get only
> 3M results, but when I run query using hbase shell I get 14M.
>
> Here is hbase shell query:
> /scan 'mgrs', {COLUMNS => 'mtf:fences',FILTER =>
>
> org.apache.hadoop.hbase.filter.RowFilter.new(CompareFilter::CompareOp.valueOf('EQUAL'),RegexStringComparator.new("._1"))}/
>
> Why my pagination Java query returns less results than hbase shell query?
> Maybe I miss some configuration on client side?
>
> thanks.
>
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/Pagination-using-Java-API-wrong-behaviour-tp4068842.html
> Sent from the HBase User mailing list archive at Nabble.com.
>