Excessive filtering in RegionScannerImpl

[Vladimir Rodionov <vl...@gmail.com>]

I may be be do not understand fully the overall design of a scanner system
in HBase but it looks like we do double filtering on KVs during scan
operation. First time in StoreScanner.next (see ScanQueryMatcher.match) and
second time in RegionScannerImpl.nextInternal.

-Vladimir

Re: Excessive filtering in RegionScannerImpl

[lars hofhansl <la...@apache.org>]

Filters can work at the row level and the KV level.
In RegionScannerImpl.nextInternal we do the row level filtering and in StoreScanner.next() via ScanQueryMatcher we do the KV based filtering.
So I would not call this excessive, but rather that we are doing the filtering at the appropriate level. I'm sure that can be improved perf wise, though.

-- Lars



----- Original Message -----
From: Stack <st...@duboce.net>
To: HBase Dev List <de...@hbase.apache.org>
Cc: 
Sent: Saturday, October 12, 2013 1:55 PM
Subject: Re: Excessive filtering in RegionScannerImpl

On Sat, Oct 12, 2013 at 11:25 AM, Vladimir Rodionov

<vl...@gmail.com>wrote:

> I may be be do not understand fully the overall design of a scanner system
> in HBase but it looks like we do double filtering on KVs during scan
> operation. First time in StoreScanner.next (see ScanQueryMatcher.match) and
> second time in RegionScannerImpl.nextInternal.
>


Quiet possible.  LarsH?
Thanks for looking into this Vladimir.
St.Ack

Re: Excessive filtering in RegionScannerImpl

[Stack <st...@duboce.net>]

On Sat, Oct 12, 2013 at 11:25 AM, Vladimir Rodionov
<vl...@gmail.com>wrote:

> I may be be do not understand fully the overall design of a scanner system
> in HBase but it looks like we do double filtering on KVs during scan
> operation. First time in StoreScanner.next (see ScanQueryMatcher.match) and
> second time in RegionScannerImpl.nextInternal.
>


Quiet possible.  LarsH?
Thanks for looking into this Vladimir.
St.Ack