You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by NNever <nn...@gmail.com> on 2012/02/21 14:28:36 UTC
HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Hi~
One customFilter, Override filterKeyValue(KeyValue v).
when the filter filterKeyValue a row's first keyValue, it will return
"ReturnCode.NEXT_ROW" to jump to next row.
But what infact is, the result changes when there are more than one
columnFamily:(here are some logs)
[filterRowKey] PERSONA1
[filterKeyValue] family:info | qualifier:active | value:\x00
[filterKeyValue] returnCode is NEXT_ROW
[reset]
[filterRowKey] PERSONA2
[filterKeyValue] family:info | qualifier:active | value:\x00
[filterKeyValue] returnCode is NEXT_ROW
[reset]
[filterRowKey] PERSONA3
[filterKeyValue] family:info | qualifier:active | value:\x00
[filterKeyValue] returnCode is NEXT_ROW
[filterKeyValue] family:npo | qualifier:059201 | value:
[filterKeyValue] returnCode is NEXT_ROW
[reset]
[filterRowKey] PERSONA4
[filterKeyValue] family:cert | qualifier:certSN | value:
PERSONAL4314120472582094317514215676313826416149
[filterKeyValue] returnCode is NEXT_ROW
[filterKeyValue] family:info | qualifier:active | value:\x00
[filterKeyValue] returnCode is NEXT_ROW
[filterKeyValue] family:npo | qualifier:059201 | value:
[filterKeyValue] returnCode is NEXT_ROW
[reset]
the Table schema is
User
info:name, info:address, info:active.... (info family, every record has
values)
npo:xxxx.....(npo family may be not exist)
cert:certSN, cert:certType....(cert family may be not exist)
As you can see from the log, once a row has more than one family values,
The ReturnCode.NEXT_ROW will not jump to next row but pass the left
qualifier values and jump to next fmaily.
Why? It's a bug or just design to be like this?
Thanks~
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Not to drift further from the subject (actually yes), but looking over
the team page it seems that at least 50% of the committers' first
language is not english so... I guess we generally can relate to that
:)
J-D
On Fri, Mar 2, 2012 at 8:43 PM, lars hofhansl <lh...@yahoo.com> wrote:
> German might have been easier, although I really meant "natural language" as opposed to code :-)
>
>
>
> ________________________________
> From: "yuzhihong@gmail.com" <yu...@gmail.com>
> To: lars hofhansl <lh...@yahoo.com>
> Cc: "user@hbase.apache.org" <us...@hbase.apache.org>
> Sent: Friday, March 2, 2012 6:52 PM
> Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
>
>
> Thanks Lars for explaining this.
>
> Did you mean that expressing in German is easier :-)
>
>
>
>
> On Mar 2, 2012, at 6:44 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
>
> I finally looked into it. This is expected.
>>
>>
>>
>>Filters are executed (by a StoreScanner/ScanQueryMatcher) per store. We have a store per column family.
>>The important part to observe here is that there is no intrinsic order between KeyValues that only differ in the column family, that is by design so that stores can be handled in parallel (even though we do not currently do that).
>>
>>
>>The filter behaves as if every store is scanned in parallel. Each store starts in the beginning, and then each store needs to skip ahead using the filter.
>>This is why it seemed to you that NEXT_ROW only seeks to the next column family, because you see the beginning of the scan for the next column family.
>>
>>
>>
>>Makes sense? It's a bit hard to explain in English :)
>>
>>
>>
>>-- Lars
>>
>>
>>
>>
>>________________________________
>> From: NNever <nn...@gmail.com>
>>To: user@hbase.apache.org; Ted Yu <yu...@gmail.com>
>>Sent: Wednesday, February 22, 2012 7:20 PM
>>Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
>>
>>Thanks Ted, I don't know mailing list strips attachment before.
>>
>>Here is the attache:
>>
>>TestFilter.java: http://pastebin.com/zC6EF8pX
>>and the log: http://pastebin.com/RsKJSHcn
>>
>>2012/2/23 Ted Yu <yu...@gmail.com>
>>
>>> N:
>>> Can you publish your code on pastebin or somewhere ?
>>> Mailing list strips attachment.
>>>
>>> Thanks
>>>
>>>
>>> On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
>>>
>>>> Attach is my test customFilter code --- TestFilter.
>>>> It just simply extends FilterBase and do some system.out...
>>>> You can just try any Table has more than one columnFamily like below:
>>>>
>>>> *Scan scan = new Scan();*
>>>> *scan.setFilter(new
> TestFilter());*
>>>> *hTable.getScanner(scan);*
>>>>
>>>> and look the HBase's log...
>>>>
>>>> It seems there is truely a BUG here....When filterKeyValue return
>>>> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
>>>> also there is one thing strange, why the fitlerRow() not be called?
>>>>
>>>> 2012/2/21 <yu...@gmail.com>
>>>>
>>>> The javadoc says filterRow() will still be called.
>>>>>
>>>>> Can you show us your filterRow() code ?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>>>>
>>>>> > Hi~
>>>>>
> >
>>>>> > One customFilter, Override filterKeyValue(KeyValue v).
>>>>> > when the filter filterKeyValue a row's first keyValue, it will return
>>>>> > "ReturnCode.NEXT_ROW" to jump to next row.
>>>>> >
>>>>> > But what infact is, the result changes when there are more than one
>>>>> > columnFamily:(here are some logs)
>>>>> >
>>>>> > [filterRowKey] PERSONA1
>>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [reset]
>>>>> > [filterRowKey] PERSONA2
>>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [reset]
>>>>> > [filterRowKey] PERSONA3
>>>>> > [filterKeyValue] family:info |
> qualifier:active | value:\x00
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [reset]
>>>>> > [filterRowKey] PERSONA4
>>>>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>>>>> > PERSONAL4314120472582094317514215676313826416149
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>>> > [reset]
>>>>> >
>>>>> > the Table schema is
>>>>> > User
>>>>> > info:name, info:address,
> info:active.... (info family, every record has
>>>>> > values)
>>>>> > npo:xxxx.....(npo family may be not exist)
>>>>> > cert:certSN, cert:certType....(cert family may be not exist)
>>>>> >
>>>>> > As you can see from the log, once a row has more than one family
>>>>> values,
>>>>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>>>>> > qualifier values and jump to next fmaily.
>>>>> > Why? It's a bug or just design to be like this?
>>>>> > Thanks~
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Posted by lars hofhansl <lh...@yahoo.com>.
German might have been easier, although I really meant "natural language" as opposed to code :-)
________________________________
From: "yuzhihong@gmail.com" <yu...@gmail.com>
To: lars hofhansl <lh...@yahoo.com>
Cc: "user@hbase.apache.org" <us...@hbase.apache.org>
Sent: Friday, March 2, 2012 6:52 PM
Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Thanks Lars for explaining this.
Did you mean that expressing in German is easier :-)
On Mar 2, 2012, at 6:44 PM, lars hofhansl <lh...@yahoo.com> wrote:
I finally looked into it. This is expected.
>
>
>
>Filters are executed (by a StoreScanner/ScanQueryMatcher) per store. We have a store per column family.
>The important part to observe here is that there is no intrinsic order between KeyValues that only differ in the column family, that is by design so that stores can be handled in parallel (even though we do not currently do that).
>
>
>The filter behaves as if every store is scanned in parallel. Each store starts in the beginning, and then each store needs to skip ahead using the filter.
>This is why it seemed to you that NEXT_ROW only seeks to the next column family, because you see the beginning of the scan for the next column family.
>
>
>
>Makes sense? It's a bit hard to explain in English :)
>
>
>
>-- Lars
>
>
>
>
>________________________________
> From: NNever <nn...@gmail.com>
>To: user@hbase.apache.org; Ted Yu <yu...@gmail.com>
>Sent: Wednesday, February 22, 2012 7:20 PM
>Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
>
>Thanks Ted, I don't know mailing list strips attachment before.
>
>Here is the attache:
>
>TestFilter.java: http://pastebin.com/zC6EF8pX
>and the log: http://pastebin.com/RsKJSHcn
>
>2012/2/23 Ted Yu <yu...@gmail.com>
>
>> N:
>> Can you publish your code on pastebin or somewhere ?
>> Mailing list strips attachment.
>>
>> Thanks
>>
>>
>> On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
>>
>>> Attach is my test customFilter code --- TestFilter.
>>> It just simply extends FilterBase and do some system.out...
>>> You can just try any Table has more than one columnFamily like below:
>>>
>>> *Scan scan = new Scan();*
>>> *scan.setFilter(new
TestFilter());*
>>> *hTable.getScanner(scan);*
>>>
>>> and look the HBase's log...
>>>
>>> It seems there is truely a BUG here....When filterKeyValue return
>>> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
>>> also there is one thing strange, why the fitlerRow() not be called?
>>>
>>> 2012/2/21 <yu...@gmail.com>
>>>
>>> The javadoc says filterRow() will still be called.
>>>>
>>>> Can you show us your filterRow() code ?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>>>
>>>> > Hi~
>>>>
>
>>>> > One customFilter, Override filterKeyValue(KeyValue v).
>>>> > when the filter filterKeyValue a row's first keyValue, it will return
>>>> > "ReturnCode.NEXT_ROW" to jump to next row.
>>>> >
>>>> > But what infact is, the result changes when there are more than one
>>>> > columnFamily:(here are some logs)
>>>> >
>>>> > [filterRowKey] PERSONA1
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA2
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA3
>>>> > [filterKeyValue] family:info |
qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA4
>>>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>>>> > PERSONAL4314120472582094317514215676313826416149
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> >
>>>> > the Table schema is
>>>> > User
>>>> > info:name, info:address,
info:active.... (info family, every record has
>>>> > values)
>>>> > npo:xxxx.....(npo family may be not exist)
>>>> > cert:certSN, cert:certType....(cert family may be not exist)
>>>> >
>>>> > As you can see from the log, once a row has more than one family
>>>> values,
>>>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>>>> > qualifier values and jump to next fmaily.
>>>> > Why? It's a bug or just design to be like this?
>>>> > Thanks~
>>>>
>>>
>>>
>>
>
>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Posted by yu...@gmail.com.
Thanks Lars for explaining this.
Did you mean that expressing in German is easier :-)
On Mar 2, 2012, at 6:44 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I finally looked into it. This is expected.
>
> Filters are executed (by a StoreScanner/ScanQueryMatcher) per store. We have a store per column family.
> The important part to observe here is that there is no intrinsic order between KeyValues that only differ in the column family, that is by design so that stores can be handled in parallel (even though we do not currently do that).
>
> The filter behaves as if every store is scanned in parallel. Each store starts in the beginning, and then each store needs to skip ahead using the filter.
> This is why it seemed to you that NEXT_ROW only seeks to the next column family, because you see the beginning of the scan for the next column family.
>
> Makes sense? It's a bit hard to explain in English :)
>
> -- Lars
>
> From: NNever <nn...@gmail.com>
> To: user@hbase.apache.org; Ted Yu <yu...@gmail.com>
> Sent: Wednesday, February 22, 2012 7:20 PM
> Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
>
> Thanks Ted, I don't know mailing list strips attachment before.
>
> Here is the attache:
>
> TestFilter.java: http://pastebin.com/zC6EF8pX
> and the log: http://pastebin.com/RsKJSHcn
>
> 2012/2/23 Ted Yu <yu...@gmail.com>
>
> > N:
> > Can you publish your code on pastebin or somewhere ?
> > Mailing list strips attachment.
> >
> > Thanks
> >
> >
> > On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
> >
> >> Attach is my test customFilter code --- TestFilter.
> >> It just simply extends FilterBase and do some system.out...
> >> You can just try any Table has more than one columnFamily like below:
> >>
> >> *Scan scan = new Scan();*
> >> *scan.setFilter(new TestFilter());*
> >> *hTable.getScanner(scan);*
> >>
> >> and look the HBase's log...
> >>
> >> It seems there is truely a BUG here....When filterKeyValue return
> >> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
> >> also there is one thing strange, why the fitlerRow() not be called?
> >>
> >> 2012/2/21 <yu...@gmail.com>
> >>
> >> The javadoc says filterRow() will still be called.
> >>>
> >>> Can you show us your filterRow() code ?
> >>>
> >>> Thanks
> >>>
> >>>
> >>>
> >>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
> >>>
> >>> > Hi~
> >>> >
> >>> > One customFilter, Override filterKeyValue(KeyValue v).
> >>> > when the filter filterKeyValue a row's first keyValue, it will return
> >>> > "ReturnCode.NEXT_ROW" to jump to next row.
> >>> >
> >>> > But what infact is, the result changes when there are more than one
> >>> > columnFamily:(here are some logs)
> >>> >
> >>> > [filterRowKey] PERSONA1
> >>> > [filterKeyValue] family:info | qualifier:active | value:\x00
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [reset]
> >>> > [filterRowKey] PERSONA2
> >>> > [filterKeyValue] family:info | qualifier:active | value:\x00
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [reset]
> >>> > [filterRowKey] PERSONA3
> >>> > [filterKeyValue] family:info | qualifier:active | value:\x00
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [filterKeyValue] family:npo | qualifier:059201 | value:
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [reset]
> >>> > [filterRowKey] PERSONA4
> >>> > [filterKeyValue] family:cert | qualifier:certSN | value:
> >>> > PERSONAL4314120472582094317514215676313826416149
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [filterKeyValue] family:info | qualifier:active | value:\x00
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [filterKeyValue] family:npo | qualifier:059201 | value:
> >>> > [filterKeyValue] returnCode is NEXT_ROW
> >>> > [reset]
> >>> >
> >>> > the Table schema is
> >>> > User
> >>> > info:name, info:address, info:active.... (info family, every record has
> >>> > values)
> >>> > npo:xxxx.....(npo family may be not exist)
> >>> > cert:certSN, cert:certType....(cert family may be not exist)
> >>> >
> >>> > As you can see from the log, once a row has more than one family
> >>> values,
> >>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
> >>> > qualifier values and jump to next fmaily.
> >>> > Why? It's a bug or just design to be like this?
> >>> > Thanks~
> >>>
> >>
> >>
> >
>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Posted by lars hofhansl <lh...@yahoo.com>.
I finally looked into it. This is expected.
Filters are executed (by a StoreScanner/ScanQueryMatcher) per store. We have a store per column family.
The important part to observe here is that there is no intrinsic order between KeyValues that only differ in the column family, that is by design so that stores can be handled in parallel (even though we do not currently do that).
The filter behaves as if every store is scanned in parallel. Each store starts in the beginning, and then each store needs to skip ahead using the filter.
This is why it seemed to you that NEXT_ROW only seeks to the next column family, because you see the beginning of the scan for the next column family.
Makes sense? It's a bit hard to explain in English :)
-- Lars
________________________________
From: NNever <nn...@gmail.com>
To: user@hbase.apache.org; Ted Yu <yu...@gmail.com>
Sent: Wednesday, February 22, 2012 7:20 PM
Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Thanks Ted, I don't know mailing list strips attachment before.
Here is the attache:
TestFilter.java: http://pastebin.com/zC6EF8pX
and the log: http://pastebin.com/RsKJSHcn
2012/2/23 Ted Yu <yu...@gmail.com>
> N:
> Can you publish your code on pastebin or somewhere ?
> Mailing list strips attachment.
>
> Thanks
>
>
> On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
>
>> Attach is my test customFilter code --- TestFilter.
>> It just simply extends FilterBase and do some system.out...
>> You can just try any Table has more than one columnFamily like below:
>>
>> *Scan scan = new Scan();*
>> *scan.setFilter(new TestFilter());*
>> *hTable.getScanner(scan);*
>>
>> and look the HBase's log...
>>
>> It seems there is truely a BUG here....When filterKeyValue return
>> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
>> also there is one thing strange, why the fitlerRow() not be called?
>>
>> 2012/2/21 <yu...@gmail.com>
>>
>> The javadoc says filterRow() will still be called.
>>>
>>> Can you show us your filterRow() code ?
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>>
>>> > Hi~
>>> >
>>> > One customFilter, Override filterKeyValue(KeyValue v).
>>> > when the filter filterKeyValue a row's first keyValue, it will return
>>> > "ReturnCode.NEXT_ROW" to jump to next row.
>>> >
>>> > But what infact is, the result changes when there are more than one
>>> > columnFamily:(here are some logs)
>>> >
>>> > [filterRowKey] PERSONA1
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA2
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA3
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA4
>>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>>> > PERSONAL4314120472582094317514215676313826416149
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> >
>>> > the Table schema is
>>> > User
>>> > info:name, info:address, info:active.... (info family, every record has
>>> > values)
>>> > npo:xxxx.....(npo family may be not exist)
>>> > cert:certSN, cert:certType....(cert family may be not exist)
>>> >
>>> > As you can see from the log, once a row has more than one family
>>> values,
>>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>>> > qualifier values and jump to next fmaily.
>>> > Why? It's a bug or just design to be like this?
>>> > Thanks~
>>>
>>
>>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Posted by NNever <nn...@gmail.com>.
Thanks Ted, I don't know mailing list strips attachment before.
Here is the attache:
TestFilter.java: http://pastebin.com/zC6EF8pX
and the log: http://pastebin.com/RsKJSHcn
2012/2/23 Ted Yu <yu...@gmail.com>
> N:
> Can you publish your code on pastebin or somewhere ?
> Mailing list strips attachment.
>
> Thanks
>
>
> On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
>
>> Attach is my test customFilter code --- TestFilter.
>> It just simply extends FilterBase and do some system.out...
>> You can just try any Table has more than one columnFamily like below:
>>
>> *Scan scan = new Scan();*
>> *scan.setFilter(new TestFilter());*
>> *hTable.getScanner(scan);*
>>
>> and look the HBase's log...
>>
>> It seems there is truely a BUG here....When filterKeyValue return
>> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
>> also there is one thing strange, why the fitlerRow() not be called?
>>
>> 2012/2/21 <yu...@gmail.com>
>>
>> The javadoc says filterRow() will still be called.
>>>
>>> Can you show us your filterRow() code ?
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>>
>>> > Hi~
>>> >
>>> > One customFilter, Override filterKeyValue(KeyValue v).
>>> > when the filter filterKeyValue a row's first keyValue, it will return
>>> > "ReturnCode.NEXT_ROW" to jump to next row.
>>> >
>>> > But what infact is, the result changes when there are more than one
>>> > columnFamily:(here are some logs)
>>> >
>>> > [filterRowKey] PERSONA1
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA2
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA3
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> > [filterRowKey] PERSONA4
>>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>>> > PERSONAL4314120472582094317514215676313826416149
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>> > [filterKeyValue] returnCode is NEXT_ROW
>>> > [reset]
>>> >
>>> > the Table schema is
>>> > User
>>> > info:name, info:address, info:active.... (info family, every record has
>>> > values)
>>> > npo:xxxx.....(npo family may be not exist)
>>> > cert:certSN, cert:certType....(cert family may be not exist)
>>> >
>>> > As you can see from the log, once a row has more than one family
>>> values,
>>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>>> > qualifier values and jump to next fmaily.
>>> > Why? It's a bug or just design to be like this?
>>> > Thanks~
>>>
>>
>>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Posted by Ted Yu <yu...@gmail.com>.
N:
Can you publish your code on pastebin or somewhere ?
Mailing list strips attachment.
Thanks
On Tue, Feb 21, 2012 at 5:47 PM, NNever <nn...@gmail.com> wrote:
> Attach is my test customFilter code --- TestFilter.
> It just simply extends FilterBase and do some system.out...
> You can just try any Table has more than one columnFamily like below:
>
> *Scan scan = new Scan();*
> *scan.setFilter(new TestFilter());*
> *hTable.getScanner(scan);*
>
> and look the HBase's log...
>
> It seems there is truely a BUG here....When filterKeyValue return
> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
> also there is one thing strange, why the fitlerRow() not be called?
>
> 2012/2/21 <yu...@gmail.com>
>
> The javadoc says filterRow() will still be called.
>>
>> Can you show us your filterRow() code ?
>>
>> Thanks
>>
>>
>>
>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>
>> > Hi~
>> >
>> > One customFilter, Override filterKeyValue(KeyValue v).
>> > when the filter filterKeyValue a row's first keyValue, it will return
>> > "ReturnCode.NEXT_ROW" to jump to next row.
>> >
>> > But what infact is, the result changes when there are more than one
>> > columnFamily:(here are some logs)
>> >
>> > [filterRowKey] PERSONA1
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA2
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA3
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA4
>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>> > PERSONAL4314120472582094317514215676313826416149
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> >
>> > the Table schema is
>> > User
>> > info:name, info:address, info:active.... (info family, every record has
>> > values)
>> > npo:xxxx.....(npo family may be not exist)
>> > cert:certSN, cert:certType....(cert family may be not exist)
>> >
>> > As you can see from the log, once a row has more than one family values,
>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>> > qualifier values and jump to next fmaily.
>> > Why? It's a bug or just design to be like this?
>> > Thanks~
>>
>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Posted by NNever <nn...@gmail.com>.
Anyone got time to make a test for this?
It really confuse me....
2012/2/22 NNever <nn...@gmail.com>
> Attach is my test customFilter code --- TestFilter.
> It just simply extends FilterBase and do some system.out...
> You can just try any Table has more than one columnFamily like below:
>
> *Scan scan = new Scan();*
> *scan.setFilter(new TestFilter());*
> *hTable.getScanner(scan);*
>
> and look the HBase's log...
>
> It seems there is truely a BUG here....When filterKeyValue return
> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
> also there is one thing strange, why the fitlerRow() not be called?
>
> 2012/2/21 <yu...@gmail.com>
>
> The javadoc says filterRow() will still be called.
>>
>> Can you show us your filterRow() code ?
>>
>> Thanks
>>
>>
>>
>> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>>
>> > Hi~
>> >
>> > One customFilter, Override filterKeyValue(KeyValue v).
>> > when the filter filterKeyValue a row's first keyValue, it will return
>> > "ReturnCode.NEXT_ROW" to jump to next row.
>> >
>> > But what infact is, the result changes when there are more than one
>> > columnFamily:(here are some logs)
>> >
>> > [filterRowKey] PERSONA1
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA2
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA3
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> > [filterRowKey] PERSONA4
>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>> > PERSONAL4314120472582094317514215676313826416149
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>> > [filterKeyValue] returnCode is NEXT_ROW
>> > [reset]
>> >
>> > the Table schema is
>> > User
>> > info:name, info:address, info:active.... (info family, every record has
>> > values)
>> > npo:xxxx.....(npo family may be not exist)
>> > cert:certSN, cert:certType....(cert family may be not exist)
>> >
>> > As you can see from the log, once a row has more than one family values,
>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>> > qualifier values and jump to next fmaily.
>> > Why? It's a bug or just design to be like this?
>> > Thanks~
>>
>
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next
columnFamily but not next row?
Posted by NNever <nn...@gmail.com>.
Attach is my test customFilter code --- TestFilter.
It just simply extends FilterBase and do some system.out...
You can just try any Table has more than one columnFamily like below:
*Scan scan = new Scan();*
*scan.setFilter(new TestFilter());*
*hTable.getScanner(scan);*
and look the HBase's log...
It seems there is truely a BUG here....When filterKeyValue return
ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
also there is one thing strange, why the fitlerRow() not be called?
2012/2/21 <yu...@gmail.com>
> The javadoc says filterRow() will still be called.
>
> Can you show us your filterRow() code ?
>
> Thanks
>
>
>
> On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
>
> > Hi~
> >
> > One customFilter, Override filterKeyValue(KeyValue v).
> > when the filter filterKeyValue a row's first keyValue, it will return
> > "ReturnCode.NEXT_ROW" to jump to next row.
> >
> > But what infact is, the result changes when there are more than one
> > columnFamily:(here are some logs)
> >
> > [filterRowKey] PERSONA1
> > [filterKeyValue] family:info | qualifier:active | value:\x00
> > [filterKeyValue] returnCode is NEXT_ROW
> > [reset]
> > [filterRowKey] PERSONA2
> > [filterKeyValue] family:info | qualifier:active | value:\x00
> > [filterKeyValue] returnCode is NEXT_ROW
> > [reset]
> > [filterRowKey] PERSONA3
> > [filterKeyValue] family:info | qualifier:active | value:\x00
> > [filterKeyValue] returnCode is NEXT_ROW
> > [filterKeyValue] family:npo | qualifier:059201 | value:
> > [filterKeyValue] returnCode is NEXT_ROW
> > [reset]
> > [filterRowKey] PERSONA4
> > [filterKeyValue] family:cert | qualifier:certSN | value:
> > PERSONAL4314120472582094317514215676313826416149
> > [filterKeyValue] returnCode is NEXT_ROW
> > [filterKeyValue] family:info | qualifier:active | value:\x00
> > [filterKeyValue] returnCode is NEXT_ROW
> > [filterKeyValue] family:npo | qualifier:059201 | value:
> > [filterKeyValue] returnCode is NEXT_ROW
> > [reset]
> >
> > the Table schema is
> > User
> > info:name, info:address, info:active.... (info family, every record has
> > values)
> > npo:xxxx.....(npo family may be not exist)
> > cert:certSN, cert:certType....(cert family may be not exist)
> >
> > As you can see from the log, once a row has more than one family values,
> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
> > qualifier values and jump to next fmaily.
> > Why? It's a bug or just design to be like this?
> > Thanks~
>
Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Posted by yu...@gmail.com.
The javadoc says filterRow() will still be called.
Can you show us your filterRow() code ?
Thanks
On Feb 21, 2012, at 7:28 AM, NNever <nn...@gmail.com> wrote:
> Hi~
>
> One customFilter, Override filterKeyValue(KeyValue v).
> when the filter filterKeyValue a row's first keyValue, it will return
> "ReturnCode.NEXT_ROW" to jump to next row.
>
> But what infact is, the result changes when there are more than one
> columnFamily:(here are some logs)
>
> [filterRowKey] PERSONA1
> [filterKeyValue] family:info | qualifier:active | value:\x00
> [filterKeyValue] returnCode is NEXT_ROW
> [reset]
> [filterRowKey] PERSONA2
> [filterKeyValue] family:info | qualifier:active | value:\x00
> [filterKeyValue] returnCode is NEXT_ROW
> [reset]
> [filterRowKey] PERSONA3
> [filterKeyValue] family:info | qualifier:active | value:\x00
> [filterKeyValue] returnCode is NEXT_ROW
> [filterKeyValue] family:npo | qualifier:059201 | value:
> [filterKeyValue] returnCode is NEXT_ROW
> [reset]
> [filterRowKey] PERSONA4
> [filterKeyValue] family:cert | qualifier:certSN | value:
> PERSONAL4314120472582094317514215676313826416149
> [filterKeyValue] returnCode is NEXT_ROW
> [filterKeyValue] family:info | qualifier:active | value:\x00
> [filterKeyValue] returnCode is NEXT_ROW
> [filterKeyValue] family:npo | qualifier:059201 | value:
> [filterKeyValue] returnCode is NEXT_ROW
> [reset]
>
> the Table schema is
> User
> info:name, info:address, info:active.... (info family, every record has
> values)
> npo:xxxx.....(npo family may be not exist)
> cert:certSN, cert:certType....(cert family may be not exist)
>
> As you can see from the log, once a row has more than one family values,
> The ReturnCode.NEXT_ROW will not jump to next row but pass the left
> qualifier values and jump to next fmaily.
> Why? It's a bug or just design to be like this?
> Thanks~