You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Carl M <so...@hotmail.com> on 2016/12/29 20:56:56 UTC
Is it possible to implement a NOT filter in Hbase?
Hi guys
I'm trying to implement some kind of NOT filter in Hbase, but don't know if possible, I'm playing with FilterIfMissing and FilterList.Operator but without luck.
I know how to return rows not having a specific column, but I mean something like returning rows NOT fullfilling a condition, where condition could be not only a SingleColumnValueFilter but a combined condition with FilterList. In SQL would be something like this for example
SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
Thanks in advance,
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
Last line should have read:
(a != '123') OR (b != '456')
On Thu, Dec 29, 2016 at 1:10 PM, Ted Yu <yu...@gmail.com> wrote:
> You can try negating the ReturnCode from filterKeyValue() (at the root of
> FilterList):
>
> abstract public ReturnCode filterKeyValue(final Cell v) throws
> IOException;
>
> INCLUDE -> SKIP
>
> SKIP -> INCLUDE
>
> Alternatively, you can use De Morgan's law to transfer the condition:
>
> NOT (a = '123' AND b = '456') becomes
>
> (NOT a = '123') OR (b = '456')
>
> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com> wrote:
>
>> Hi guys
>>
>>
>> I'm trying to implement some kind of NOT filter in Hbase, but don't know
>> if possible, I'm playing with FilterIfMissing and FilterList.Operator but
>> without luck.
>>
>>
>> I know how to return rows not having a specific column, but I mean
>> something like returning rows NOT fullfilling a condition, where condition
>> could be not only a SingleColumnValueFilter but a combined condition with
>> FilterList. In SQL would be something like this for example
>>
>>
>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
>>
>>
>> Thanks in advance,
>>
>
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
Your first approach is more general (beneficial to more users).
However, looks like neither of us has the bandwidth to make it work.
On Mon, Jan 9, 2017 at 11:30 AM, Carl M <so...@hotmail.com> wrote:
> Hi Ted,
>
>
> Just to tell that unfortunately i wasn't seeing any
> INCLUDE_AND_SEEK_NEXT_ROW nor INCLUDE_AND_NEXT_COL when debuging so adding
> them were not going to make any difference.
>
>
> Nevetherless i implement De Morgan's law as you second suggestion (it was
> easier as I expected) and it's working, so thanks again for that!
>
> Best,
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: lunes, 02 de enero de 2017 11:36 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> There is INCLUDE_AND_SEEK_NEXT_ROW which was not accounted for in the
> nested if statement.
>
> Is it possible for you to come up with unit test showing what you observe ?
>
> If you don't have time, please consider using De Morgan's law.
>
> Cheers
>
> On Mon, Jan 2, 2017 at 5:19 PM, Carl M <so...@hotmail.com> wrote:
>
> > Thanks for your response Ted.
> >
> >
> > I did the change, unfortunately it doesn't make any difference.
> >
> >
> > Best,
> >
> > ________________________________
> > De: Ted Yu <yu...@gmail.com>
> > Enviado: lunes, 02 de enero de 2017 07:58 p.m.
> > Para: user@hbase.apache.org
> > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >
> > In the nested if statement, can you also handle NEXT_COL return code ?
> >
> > It should be translated to INCLUDE_AND_NEXT_COL
> >
> > Cheers
> >
> > > On Jan 2, 2017, at 1:54 PM, Carl M <so...@hotmail.com> wrote:
> > >
> > > Sorry for not being clear enough.
> > >
> > >
> > > Maybe i misunderstand your suggestion, but what I've done in implement
> > my own filter, wrapping a FilterList
> > >
> > > public class NotFilterList extends Filter {
> > > private FilterList filter;
> > >
> > > public NotFilterList(final List<Filter> rowFilters) {
> > > this.filter = new FilterList(rowFilters);
> > > }
> > >
> > > public void reset() throws IOException {
> > > this.filter.reset();
> > > }
> > >
> > > public boolean filterRowKey(byte[] rowKey, int offset, int length)
> > throws IOException {
> > > return this.filter.filterRowKey(rowKey, offset, length);
> > > }
> > >
> > > public boolean filterAllRemaining() throws IOException {
> > > return this.filter.filterAllRemaining();
> > > }
> > >
> > > public ReturnCode filterKeyValue(Cell v) throws IOException {
> > > ReturnCode code = this.filter.filterKeyValue(v);
> > >
> > > if (code == ReturnCode.INCLUDE)
> > > code = ReturnCode.SKIP;
> > > else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
> > > code = ReturnCode.NEXT_COL;
> > > else if (code == ReturnCode.NEXT_ROW)
> > > code = ReturnCode.INCLUDE_AND_NEXT_COL;
> > > else if (code == ReturnCode.SKIP) {
> > > code = ReturnCode.INCLUDE;
> > > }
> > >
> > > return code;
> > > }
> > >
> > > public Cell transformCell(Cell v) throws IOException {
> > > return this.filter.transformCell(v);
> > > }
> > >
> > > public KeyValue transform(KeyValue currentKV) throws IOException {
> > > return this.filter.transform(currentKV);
> > > }
> > >
> > > public void filterRowCells(List<Cell> kvs) throws IOException {
> > > this.filter.filterRowCells(kvs);
> > > }
> > >
> > > public boolean hasFilterRow() {
> > > return this.filter.hasFilterRow();
> > > }
> > >
> > > public boolean filterRow() throws IOException {
> > > return this.filter.filterRow();
> > > }
> > >
> > > public KeyValue getNextKeyHint(KeyValue currentKV) throws
> IOException
> > {
> > > return this.filter.getNextKeyHint(currentKV);
> > > }
> > >
> > > public Cell getNextCellHint(Cell currentKV) throws IOException {
> > > return this.filter.getNextCellHint(currentKV);
> > > }
> > >
> > > public boolean isFamilyEssential(byte[] name) throws IOException {
> > > return this.filter.isFamilyEssential(name);
> > > }
> > >
> > > public byte[] toByteArray() throws IOException {
> > > return this.filter.toByteArray();
> > > }
> > >
> > > public static NotFilterList parseFrom(final byte [] pbBytes) throws
> > DeserializationException {
> > > FilterList filterList = FilterList.parseFrom(pbBytes);
> > > return new NotFilterList(filterList.getFilters());
> > > }
> > >
> > > boolean areSerializedFieldsEqual(Filter other) {
> > > return this.filter.areSerializedFieldsEqual(other);
> > > }
> > >
> > > }
> > >
> > >
> > > What I mean is that filterKeyValue in the way i have it now, return
> the
> > right results but only the fields that were not originally skipped.
> > >
> > >
> > > So for example, if i have two rows each one with two fields
> > >
> > > Row 1
> > >
> > > Name: Bill
> > >
> > > Surname: Gates
> > >
> > >
> > > Row 2
> > >
> > > Name: Steve
> > >
> > > Surname: Jobs
> > >
> > >
> > > And I want to query for Rows that doesn't have Name 'Bill'
> > >
> > > NOT (Name='Bill')
> > >
> > >
> > > What I get as result from Hbase with this NotFilter is
> > >
> > > Row 2
> > >
> > > Surname: Jobs
> > >
> > >
> > > I suppose it's related to the cell "Name: Steve" skipped in the first
> > place (before reversing the ReturnCode).
> > >
> > >
> > > Best,
> > >
> > >
> > >
> > > ________________________________
> > > De: Ted Yu <yu...@gmail.com>
> > > Enviado: lunes, 02 de enero de 2017 06:31 p.m.
> > > Para: user@hbase.apache.org
> > > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> > >
> > > bq. the cell/value that was originally skip is not return
> > >
> > > Can you be a bit more specific (with a concrete example) : the skipped
> > cell
> > > would not be returned (as indicated by the ReturnCode).
> > >
> > > Thanks
> > >
> > >> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com>
> wrote:
> > >>
> > >> Hi Ted,
> > >>
> > >>
> > >> I tried your suggestion, unfortunately it doesn't work as expected. I
> > >> don't fully understand FilterList, but if a cell value was skip and I
> > >> reverse the ReturnCode, i get the right row but the cell/value that
> was
> > >> originally skip is not return.
> > >>
> > >> I also tried reversing only filterRow() method of FilterList, but I
> got
> > >> the same behaviour (the original cell/value missing).
> > >>
> > >>
> > >> Best,
> > >>
> > >>
> > >> ________________________________
> > >> De: Ted Yu <yu...@gmail.com>
> > >> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
> > >> Para: user@hbase.apache.org
> > >> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> > >>
> > >> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL :
> > you're
> > >> not interested in any version of the current Cell.
> > >>
> > >> Cheers
> > >>
> > >>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com>
> > wrote:
> > >>>
> > >>> Thanks Ted! Great idea replacing the value in filterKeyValue.
> Although
> > >> I'm
> > >>> not quite sure looking at FilterList code if only INCLUDE/SKIP should
> > be
> > >>> replaced, and which should be the correct replacement for
> > >>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> > >>> implement DeMorgan's law but I think it would be harder.
> > >>>
> > >>>
> > >>> Best,
> > >>>
> > >>> ________________________________
> > >>> De: Ted Yu <yu...@gmail.com>
> > >>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> > >>> Para: user@hbase.apache.org
> > >>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> > >>>
> > >>> You can try negating the ReturnCode from filterKeyValue() (at the
> root
> > of
> > >>> FilterList):
> > >>>
> > >>> abstract public ReturnCode filterKeyValue(final Cell v) throws
> > >>> IOException;
> > >>>
> > >>> INCLUDE -> SKIP
> > >>>
> > >>> SKIP -> INCLUDE
> > >>>
> > >>> Alternatively, you can use De Morgan's law to transfer the condition:
> > >>>
> > >>> NOT (a = '123' AND b = '456') becomes
> > >>>
> > >>> (NOT a = '123') OR (b = '456')
> > >>>
> > >>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
> > >>> wrote:
> > >>>
> > >>>> Hi guys
> > >>>>
> > >>>>
> > >>>> I'm trying to implement some kind of NOT filter in Hbase, but don't
> > >> know
> > >>>> if possible, I'm playing with FilterIfMissing and
> FilterList.Operator
> > >> but
> > >>>> without luck.
> > >>>>
> > >>>>
> > >>>> I know how to return rows not having a specific column, but I mean
> > >>>> something like returning rows NOT fullfilling a condition, where
> > >>> condition
> > >>>> could be not only a SingleColumnValueFilter but a combined condition
> > >> with
> > >>>> FilterList. In SQL would be something like this for example
> > >>>>
> > >>>>
> > >>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> > >>>>
> > >>>>
> > >>>> Thanks in advance,
> > >>
> >
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Carl M <so...@hotmail.com>.
Hi Ted,
Just to tell that unfortunately i wasn't seeing any INCLUDE_AND_SEEK_NEXT_ROW nor INCLUDE_AND_NEXT_COL when debuging so adding them were not going to make any difference.
Nevetherless i implement De Morgan's law as you second suggestion (it was easier as I expected) and it's working, so thanks again for that!
Best,
________________________________
De: Ted Yu <yu...@gmail.com>
Enviado: lunes, 02 de enero de 2017 11:36 p.m.
Para: user@hbase.apache.org
Asunto: Re: Is it possible to implement a NOT filter in Hbase?
There is INCLUDE_AND_SEEK_NEXT_ROW which was not accounted for in the
nested if statement.
Is it possible for you to come up with unit test showing what you observe ?
If you don't have time, please consider using De Morgan's law.
Cheers
On Mon, Jan 2, 2017 at 5:19 PM, Carl M <so...@hotmail.com> wrote:
> Thanks for your response Ted.
>
>
> I did the change, unfortunately it doesn't make any difference.
>
>
> Best,
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: lunes, 02 de enero de 2017 07:58 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> In the nested if statement, can you also handle NEXT_COL return code ?
>
> It should be translated to INCLUDE_AND_NEXT_COL
>
> Cheers
>
> > On Jan 2, 2017, at 1:54 PM, Carl M <so...@hotmail.com> wrote:
> >
> > Sorry for not being clear enough.
> >
> >
> > Maybe i misunderstand your suggestion, but what I've done in implement
> my own filter, wrapping a FilterList
> >
> > public class NotFilterList extends Filter {
> > private FilterList filter;
> >
> > public NotFilterList(final List<Filter> rowFilters) {
> > this.filter = new FilterList(rowFilters);
> > }
> >
> > public void reset() throws IOException {
> > this.filter.reset();
> > }
> >
> > public boolean filterRowKey(byte[] rowKey, int offset, int length)
> throws IOException {
> > return this.filter.filterRowKey(rowKey, offset, length);
> > }
> >
> > public boolean filterAllRemaining() throws IOException {
> > return this.filter.filterAllRemaining();
> > }
> >
> > public ReturnCode filterKeyValue(Cell v) throws IOException {
> > ReturnCode code = this.filter.filterKeyValue(v);
> >
> > if (code == ReturnCode.INCLUDE)
> > code = ReturnCode.SKIP;
> > else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
> > code = ReturnCode.NEXT_COL;
> > else if (code == ReturnCode.NEXT_ROW)
> > code = ReturnCode.INCLUDE_AND_NEXT_COL;
> > else if (code == ReturnCode.SKIP) {
> > code = ReturnCode.INCLUDE;
> > }
> >
> > return code;
> > }
> >
> > public Cell transformCell(Cell v) throws IOException {
> > return this.filter.transformCell(v);
> > }
> >
> > public KeyValue transform(KeyValue currentKV) throws IOException {
> > return this.filter.transform(currentKV);
> > }
> >
> > public void filterRowCells(List<Cell> kvs) throws IOException {
> > this.filter.filterRowCells(kvs);
> > }
> >
> > public boolean hasFilterRow() {
> > return this.filter.hasFilterRow();
> > }
> >
> > public boolean filterRow() throws IOException {
> > return this.filter.filterRow();
> > }
> >
> > public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException
> {
> > return this.filter.getNextKeyHint(currentKV);
> > }
> >
> > public Cell getNextCellHint(Cell currentKV) throws IOException {
> > return this.filter.getNextCellHint(currentKV);
> > }
> >
> > public boolean isFamilyEssential(byte[] name) throws IOException {
> > return this.filter.isFamilyEssential(name);
> > }
> >
> > public byte[] toByteArray() throws IOException {
> > return this.filter.toByteArray();
> > }
> >
> > public static NotFilterList parseFrom(final byte [] pbBytes) throws
> DeserializationException {
> > FilterList filterList = FilterList.parseFrom(pbBytes);
> > return new NotFilterList(filterList.getFilters());
> > }
> >
> > boolean areSerializedFieldsEqual(Filter other) {
> > return this.filter.areSerializedFieldsEqual(other);
> > }
> >
> > }
> >
> >
> > What I mean is that filterKeyValue in the way i have it now, return the
> right results but only the fields that were not originally skipped.
> >
> >
> > So for example, if i have two rows each one with two fields
> >
> > Row 1
> >
> > Name: Bill
> >
> > Surname: Gates
> >
> >
> > Row 2
> >
> > Name: Steve
> >
> > Surname: Jobs
> >
> >
> > And I want to query for Rows that doesn't have Name 'Bill'
> >
> > NOT (Name='Bill')
> >
> >
> > What I get as result from Hbase with this NotFilter is
> >
> > Row 2
> >
> > Surname: Jobs
> >
> >
> > I suppose it's related to the cell "Name: Steve" skipped in the first
> place (before reversing the ReturnCode).
> >
> >
> > Best,
> >
> >
> >
> > ________________________________
> > De: Ted Yu <yu...@gmail.com>
> > Enviado: lunes, 02 de enero de 2017 06:31 p.m.
> > Para: user@hbase.apache.org
> > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >
> > bq. the cell/value that was originally skip is not return
> >
> > Can you be a bit more specific (with a concrete example) : the skipped
> cell
> > would not be returned (as indicated by the ReturnCode).
> >
> > Thanks
> >
> >> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
> >>
> >> Hi Ted,
> >>
> >>
> >> I tried your suggestion, unfortunately it doesn't work as expected. I
> >> don't fully understand FilterList, but if a cell value was skip and I
> >> reverse the ReturnCode, i get the right row but the cell/value that was
> >> originally skip is not return.
> >>
> >> I also tried reversing only filterRow() method of FilterList, but I got
> >> the same behaviour (the original cell/value missing).
> >>
> >>
> >> Best,
> >>
> >>
> >> ________________________________
> >> De: Ted Yu <yu...@gmail.com>
> >> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
> >> Para: user@hbase.apache.org
> >> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >>
> >> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL :
> you're
> >> not interested in any version of the current Cell.
> >>
> >> Cheers
> >>
> >>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com>
> wrote:
> >>>
> >>> Thanks Ted! Great idea replacing the value in filterKeyValue. Although
> >> I'm
> >>> not quite sure looking at FilterList code if only INCLUDE/SKIP should
> be
> >>> replaced, and which should be the correct replacement for
> >>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> >>> implement DeMorgan's law but I think it would be harder.
> >>>
> >>>
> >>> Best,
> >>>
> >>> ________________________________
> >>> De: Ted Yu <yu...@gmail.com>
> >>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> >>> Para: user@hbase.apache.org
> >>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >>>
> >>> You can try negating the ReturnCode from filterKeyValue() (at the root
> of
> >>> FilterList):
> >>>
> >>> abstract public ReturnCode filterKeyValue(final Cell v) throws
> >>> IOException;
> >>>
> >>> INCLUDE -> SKIP
> >>>
> >>> SKIP -> INCLUDE
> >>>
> >>> Alternatively, you can use De Morgan's law to transfer the condition:
> >>>
> >>> NOT (a = '123' AND b = '456') becomes
> >>>
> >>> (NOT a = '123') OR (b = '456')
> >>>
> >>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
> >>> wrote:
> >>>
> >>>> Hi guys
> >>>>
> >>>>
> >>>> I'm trying to implement some kind of NOT filter in Hbase, but don't
> >> know
> >>>> if possible, I'm playing with FilterIfMissing and FilterList.Operator
> >> but
> >>>> without luck.
> >>>>
> >>>>
> >>>> I know how to return rows not having a specific column, but I mean
> >>>> something like returning rows NOT fullfilling a condition, where
> >>> condition
> >>>> could be not only a SingleColumnValueFilter but a combined condition
> >> with
> >>>> FilterList. In SQL would be something like this for example
> >>>>
> >>>>
> >>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> >>>>
> >>>>
> >>>> Thanks in advance,
> >>
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
There is INCLUDE_AND_SEEK_NEXT_ROW which was not accounted for in the
nested if statement.
Is it possible for you to come up with unit test showing what you observe ?
If you don't have time, please consider using De Morgan's law.
Cheers
On Mon, Jan 2, 2017 at 5:19 PM, Carl M <so...@hotmail.com> wrote:
> Thanks for your response Ted.
>
>
> I did the change, unfortunately it doesn't make any difference.
>
>
> Best,
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: lunes, 02 de enero de 2017 07:58 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> In the nested if statement, can you also handle NEXT_COL return code ?
>
> It should be translated to INCLUDE_AND_NEXT_COL
>
> Cheers
>
> > On Jan 2, 2017, at 1:54 PM, Carl M <so...@hotmail.com> wrote:
> >
> > Sorry for not being clear enough.
> >
> >
> > Maybe i misunderstand your suggestion, but what I've done in implement
> my own filter, wrapping a FilterList
> >
> > public class NotFilterList extends Filter {
> > private FilterList filter;
> >
> > public NotFilterList(final List<Filter> rowFilters) {
> > this.filter = new FilterList(rowFilters);
> > }
> >
> > public void reset() throws IOException {
> > this.filter.reset();
> > }
> >
> > public boolean filterRowKey(byte[] rowKey, int offset, int length)
> throws IOException {
> > return this.filter.filterRowKey(rowKey, offset, length);
> > }
> >
> > public boolean filterAllRemaining() throws IOException {
> > return this.filter.filterAllRemaining();
> > }
> >
> > public ReturnCode filterKeyValue(Cell v) throws IOException {
> > ReturnCode code = this.filter.filterKeyValue(v);
> >
> > if (code == ReturnCode.INCLUDE)
> > code = ReturnCode.SKIP;
> > else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
> > code = ReturnCode.NEXT_COL;
> > else if (code == ReturnCode.NEXT_ROW)
> > code = ReturnCode.INCLUDE_AND_NEXT_COL;
> > else if (code == ReturnCode.SKIP) {
> > code = ReturnCode.INCLUDE;
> > }
> >
> > return code;
> > }
> >
> > public Cell transformCell(Cell v) throws IOException {
> > return this.filter.transformCell(v);
> > }
> >
> > public KeyValue transform(KeyValue currentKV) throws IOException {
> > return this.filter.transform(currentKV);
> > }
> >
> > public void filterRowCells(List<Cell> kvs) throws IOException {
> > this.filter.filterRowCells(kvs);
> > }
> >
> > public boolean hasFilterRow() {
> > return this.filter.hasFilterRow();
> > }
> >
> > public boolean filterRow() throws IOException {
> > return this.filter.filterRow();
> > }
> >
> > public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException
> {
> > return this.filter.getNextKeyHint(currentKV);
> > }
> >
> > public Cell getNextCellHint(Cell currentKV) throws IOException {
> > return this.filter.getNextCellHint(currentKV);
> > }
> >
> > public boolean isFamilyEssential(byte[] name) throws IOException {
> > return this.filter.isFamilyEssential(name);
> > }
> >
> > public byte[] toByteArray() throws IOException {
> > return this.filter.toByteArray();
> > }
> >
> > public static NotFilterList parseFrom(final byte [] pbBytes) throws
> DeserializationException {
> > FilterList filterList = FilterList.parseFrom(pbBytes);
> > return new NotFilterList(filterList.getFilters());
> > }
> >
> > boolean areSerializedFieldsEqual(Filter other) {
> > return this.filter.areSerializedFieldsEqual(other);
> > }
> >
> > }
> >
> >
> > What I mean is that filterKeyValue in the way i have it now, return the
> right results but only the fields that were not originally skipped.
> >
> >
> > So for example, if i have two rows each one with two fields
> >
> > Row 1
> >
> > Name: Bill
> >
> > Surname: Gates
> >
> >
> > Row 2
> >
> > Name: Steve
> >
> > Surname: Jobs
> >
> >
> > And I want to query for Rows that doesn't have Name 'Bill'
> >
> > NOT (Name='Bill')
> >
> >
> > What I get as result from Hbase with this NotFilter is
> >
> > Row 2
> >
> > Surname: Jobs
> >
> >
> > I suppose it's related to the cell "Name: Steve" skipped in the first
> place (before reversing the ReturnCode).
> >
> >
> > Best,
> >
> >
> >
> > ________________________________
> > De: Ted Yu <yu...@gmail.com>
> > Enviado: lunes, 02 de enero de 2017 06:31 p.m.
> > Para: user@hbase.apache.org
> > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >
> > bq. the cell/value that was originally skip is not return
> >
> > Can you be a bit more specific (with a concrete example) : the skipped
> cell
> > would not be returned (as indicated by the ReturnCode).
> >
> > Thanks
> >
> >> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
> >>
> >> Hi Ted,
> >>
> >>
> >> I tried your suggestion, unfortunately it doesn't work as expected. I
> >> don't fully understand FilterList, but if a cell value was skip and I
> >> reverse the ReturnCode, i get the right row but the cell/value that was
> >> originally skip is not return.
> >>
> >> I also tried reversing only filterRow() method of FilterList, but I got
> >> the same behaviour (the original cell/value missing).
> >>
> >>
> >> Best,
> >>
> >>
> >> ________________________________
> >> De: Ted Yu <yu...@gmail.com>
> >> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
> >> Para: user@hbase.apache.org
> >> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >>
> >> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL :
> you're
> >> not interested in any version of the current Cell.
> >>
> >> Cheers
> >>
> >>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com>
> wrote:
> >>>
> >>> Thanks Ted! Great idea replacing the value in filterKeyValue. Although
> >> I'm
> >>> not quite sure looking at FilterList code if only INCLUDE/SKIP should
> be
> >>> replaced, and which should be the correct replacement for
> >>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> >>> implement DeMorgan's law but I think it would be harder.
> >>>
> >>>
> >>> Best,
> >>>
> >>> ________________________________
> >>> De: Ted Yu <yu...@gmail.com>
> >>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> >>> Para: user@hbase.apache.org
> >>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >>>
> >>> You can try negating the ReturnCode from filterKeyValue() (at the root
> of
> >>> FilterList):
> >>>
> >>> abstract public ReturnCode filterKeyValue(final Cell v) throws
> >>> IOException;
> >>>
> >>> INCLUDE -> SKIP
> >>>
> >>> SKIP -> INCLUDE
> >>>
> >>> Alternatively, you can use De Morgan's law to transfer the condition:
> >>>
> >>> NOT (a = '123' AND b = '456') becomes
> >>>
> >>> (NOT a = '123') OR (b = '456')
> >>>
> >>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
> >>> wrote:
> >>>
> >>>> Hi guys
> >>>>
> >>>>
> >>>> I'm trying to implement some kind of NOT filter in Hbase, but don't
> >> know
> >>>> if possible, I'm playing with FilterIfMissing and FilterList.Operator
> >> but
> >>>> without luck.
> >>>>
> >>>>
> >>>> I know how to return rows not having a specific column, but I mean
> >>>> something like returning rows NOT fullfilling a condition, where
> >>> condition
> >>>> could be not only a SingleColumnValueFilter but a combined condition
> >> with
> >>>> FilterList. In SQL would be something like this for example
> >>>>
> >>>>
> >>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> >>>>
> >>>>
> >>>> Thanks in advance,
> >>
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Carl M <so...@hotmail.com>.
Thanks for your response Ted.
I did the change, unfortunately it doesn't make any difference.
Best,
________________________________
De: Ted Yu <yu...@gmail.com>
Enviado: lunes, 02 de enero de 2017 07:58 p.m.
Para: user@hbase.apache.org
Asunto: Re: Is it possible to implement a NOT filter in Hbase?
In the nested if statement, can you also handle NEXT_COL return code ?
It should be translated to INCLUDE_AND_NEXT_COL
Cheers
> On Jan 2, 2017, at 1:54 PM, Carl M <so...@hotmail.com> wrote:
>
> Sorry for not being clear enough.
>
>
> Maybe i misunderstand your suggestion, but what I've done in implement my own filter, wrapping a FilterList
>
> public class NotFilterList extends Filter {
> private FilterList filter;
>
> public NotFilterList(final List<Filter> rowFilters) {
> this.filter = new FilterList(rowFilters);
> }
>
> public void reset() throws IOException {
> this.filter.reset();
> }
>
> public boolean filterRowKey(byte[] rowKey, int offset, int length) throws IOException {
> return this.filter.filterRowKey(rowKey, offset, length);
> }
>
> public boolean filterAllRemaining() throws IOException {
> return this.filter.filterAllRemaining();
> }
>
> public ReturnCode filterKeyValue(Cell v) throws IOException {
> ReturnCode code = this.filter.filterKeyValue(v);
>
> if (code == ReturnCode.INCLUDE)
> code = ReturnCode.SKIP;
> else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
> code = ReturnCode.NEXT_COL;
> else if (code == ReturnCode.NEXT_ROW)
> code = ReturnCode.INCLUDE_AND_NEXT_COL;
> else if (code == ReturnCode.SKIP) {
> code = ReturnCode.INCLUDE;
> }
>
> return code;
> }
>
> public Cell transformCell(Cell v) throws IOException {
> return this.filter.transformCell(v);
> }
>
> public KeyValue transform(KeyValue currentKV) throws IOException {
> return this.filter.transform(currentKV);
> }
>
> public void filterRowCells(List<Cell> kvs) throws IOException {
> this.filter.filterRowCells(kvs);
> }
>
> public boolean hasFilterRow() {
> return this.filter.hasFilterRow();
> }
>
> public boolean filterRow() throws IOException {
> return this.filter.filterRow();
> }
>
> public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException {
> return this.filter.getNextKeyHint(currentKV);
> }
>
> public Cell getNextCellHint(Cell currentKV) throws IOException {
> return this.filter.getNextCellHint(currentKV);
> }
>
> public boolean isFamilyEssential(byte[] name) throws IOException {
> return this.filter.isFamilyEssential(name);
> }
>
> public byte[] toByteArray() throws IOException {
> return this.filter.toByteArray();
> }
>
> public static NotFilterList parseFrom(final byte [] pbBytes) throws DeserializationException {
> FilterList filterList = FilterList.parseFrom(pbBytes);
> return new NotFilterList(filterList.getFilters());
> }
>
> boolean areSerializedFieldsEqual(Filter other) {
> return this.filter.areSerializedFieldsEqual(other);
> }
>
> }
>
>
> What I mean is that filterKeyValue in the way i have it now, return the right results but only the fields that were not originally skipped.
>
>
> So for example, if i have two rows each one with two fields
>
> Row 1
>
> Name: Bill
>
> Surname: Gates
>
>
> Row 2
>
> Name: Steve
>
> Surname: Jobs
>
>
> And I want to query for Rows that doesn't have Name 'Bill'
>
> NOT (Name='Bill')
>
>
> What I get as result from Hbase with this NotFilter is
>
> Row 2
>
> Surname: Jobs
>
>
> I suppose it's related to the cell "Name: Steve" skipped in the first place (before reversing the ReturnCode).
>
>
> Best,
>
>
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: lunes, 02 de enero de 2017 06:31 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> bq. the cell/value that was originally skip is not return
>
> Can you be a bit more specific (with a concrete example) : the skipped cell
> would not be returned (as indicated by the ReturnCode).
>
> Thanks
>
>> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
>>
>> Hi Ted,
>>
>>
>> I tried your suggestion, unfortunately it doesn't work as expected. I
>> don't fully understand FilterList, but if a cell value was skip and I
>> reverse the ReturnCode, i get the right row but the cell/value that was
>> originally skip is not return.
>>
>> I also tried reversing only filterRow() method of FilterList, but I got
>> the same behaviour (the original cell/value missing).
>>
>>
>> Best,
>>
>>
>> ________________________________
>> De: Ted Yu <yu...@gmail.com>
>> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
>> Para: user@hbase.apache.org
>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>>
>> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
>> not interested in any version of the current Cell.
>>
>> Cheers
>>
>>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
>>>
>>> Thanks Ted! Great idea replacing the value in filterKeyValue. Although
>> I'm
>>> not quite sure looking at FilterList code if only INCLUDE/SKIP should be
>>> replaced, and which should be the correct replacement for
>>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
>>> implement DeMorgan's law but I think it would be harder.
>>>
>>>
>>> Best,
>>>
>>> ________________________________
>>> De: Ted Yu <yu...@gmail.com>
>>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
>>> Para: user@hbase.apache.org
>>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>>>
>>> You can try negating the ReturnCode from filterKeyValue() (at the root of
>>> FilterList):
>>>
>>> abstract public ReturnCode filterKeyValue(final Cell v) throws
>>> IOException;
>>>
>>> INCLUDE -> SKIP
>>>
>>> SKIP -> INCLUDE
>>>
>>> Alternatively, you can use De Morgan's law to transfer the condition:
>>>
>>> NOT (a = '123' AND b = '456') becomes
>>>
>>> (NOT a = '123') OR (b = '456')
>>>
>>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
>>> wrote:
>>>
>>>> Hi guys
>>>>
>>>>
>>>> I'm trying to implement some kind of NOT filter in Hbase, but don't
>> know
>>>> if possible, I'm playing with FilterIfMissing and FilterList.Operator
>> but
>>>> without luck.
>>>>
>>>>
>>>> I know how to return rows not having a specific column, but I mean
>>>> something like returning rows NOT fullfilling a condition, where
>>> condition
>>>> could be not only a SingleColumnValueFilter but a combined condition
>> with
>>>> FilterList. In SQL would be something like this for example
>>>>
>>>>
>>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
>>>>
>>>>
>>>> Thanks in advance,
>>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
In the nested if statement, can you also handle NEXT_COL return code ?
It should be translated to INCLUDE_AND_NEXT_COL
Cheers
> On Jan 2, 2017, at 1:54 PM, Carl M <so...@hotmail.com> wrote:
>
> Sorry for not being clear enough.
>
>
> Maybe i misunderstand your suggestion, but what I've done in implement my own filter, wrapping a FilterList
>
> public class NotFilterList extends Filter {
> private FilterList filter;
>
> public NotFilterList(final List<Filter> rowFilters) {
> this.filter = new FilterList(rowFilters);
> }
>
> public void reset() throws IOException {
> this.filter.reset();
> }
>
> public boolean filterRowKey(byte[] rowKey, int offset, int length) throws IOException {
> return this.filter.filterRowKey(rowKey, offset, length);
> }
>
> public boolean filterAllRemaining() throws IOException {
> return this.filter.filterAllRemaining();
> }
>
> public ReturnCode filterKeyValue(Cell v) throws IOException {
> ReturnCode code = this.filter.filterKeyValue(v);
>
> if (code == ReturnCode.INCLUDE)
> code = ReturnCode.SKIP;
> else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
> code = ReturnCode.NEXT_COL;
> else if (code == ReturnCode.NEXT_ROW)
> code = ReturnCode.INCLUDE_AND_NEXT_COL;
> else if (code == ReturnCode.SKIP) {
> code = ReturnCode.INCLUDE;
> }
>
> return code;
> }
>
> public Cell transformCell(Cell v) throws IOException {
> return this.filter.transformCell(v);
> }
>
> public KeyValue transform(KeyValue currentKV) throws IOException {
> return this.filter.transform(currentKV);
> }
>
> public void filterRowCells(List<Cell> kvs) throws IOException {
> this.filter.filterRowCells(kvs);
> }
>
> public boolean hasFilterRow() {
> return this.filter.hasFilterRow();
> }
>
> public boolean filterRow() throws IOException {
> return this.filter.filterRow();
> }
>
> public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException {
> return this.filter.getNextKeyHint(currentKV);
> }
>
> public Cell getNextCellHint(Cell currentKV) throws IOException {
> return this.filter.getNextCellHint(currentKV);
> }
>
> public boolean isFamilyEssential(byte[] name) throws IOException {
> return this.filter.isFamilyEssential(name);
> }
>
> public byte[] toByteArray() throws IOException {
> return this.filter.toByteArray();
> }
>
> public static NotFilterList parseFrom(final byte [] pbBytes) throws DeserializationException {
> FilterList filterList = FilterList.parseFrom(pbBytes);
> return new NotFilterList(filterList.getFilters());
> }
>
> boolean areSerializedFieldsEqual(Filter other) {
> return this.filter.areSerializedFieldsEqual(other);
> }
>
> }
>
>
> What I mean is that filterKeyValue in the way i have it now, return the right results but only the fields that were not originally skipped.
>
>
> So for example, if i have two rows each one with two fields
>
> Row 1
>
> Name: Bill
>
> Surname: Gates
>
>
> Row 2
>
> Name: Steve
>
> Surname: Jobs
>
>
> And I want to query for Rows that doesn't have Name 'Bill'
>
> NOT (Name='Bill')
>
>
> What I get as result from Hbase with this NotFilter is
>
> Row 2
>
> Surname: Jobs
>
>
> I suppose it's related to the cell "Name: Steve" skipped in the first place (before reversing the ReturnCode).
>
>
> Best,
>
>
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: lunes, 02 de enero de 2017 06:31 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> bq. the cell/value that was originally skip is not return
>
> Can you be a bit more specific (with a concrete example) : the skipped cell
> would not be returned (as indicated by the ReturnCode).
>
> Thanks
>
>> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
>>
>> Hi Ted,
>>
>>
>> I tried your suggestion, unfortunately it doesn't work as expected. I
>> don't fully understand FilterList, but if a cell value was skip and I
>> reverse the ReturnCode, i get the right row but the cell/value that was
>> originally skip is not return.
>>
>> I also tried reversing only filterRow() method of FilterList, but I got
>> the same behaviour (the original cell/value missing).
>>
>>
>> Best,
>>
>>
>> ________________________________
>> De: Ted Yu <yu...@gmail.com>
>> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
>> Para: user@hbase.apache.org
>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>>
>> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
>> not interested in any version of the current Cell.
>>
>> Cheers
>>
>>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
>>>
>>> Thanks Ted! Great idea replacing the value in filterKeyValue. Although
>> I'm
>>> not quite sure looking at FilterList code if only INCLUDE/SKIP should be
>>> replaced, and which should be the correct replacement for
>>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
>>> implement DeMorgan's law but I think it would be harder.
>>>
>>>
>>> Best,
>>>
>>> ________________________________
>>> De: Ted Yu <yu...@gmail.com>
>>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
>>> Para: user@hbase.apache.org
>>> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>>>
>>> You can try negating the ReturnCode from filterKeyValue() (at the root of
>>> FilterList):
>>>
>>> abstract public ReturnCode filterKeyValue(final Cell v) throws
>>> IOException;
>>>
>>> INCLUDE -> SKIP
>>>
>>> SKIP -> INCLUDE
>>>
>>> Alternatively, you can use De Morgan's law to transfer the condition:
>>>
>>> NOT (a = '123' AND b = '456') becomes
>>>
>>> (NOT a = '123') OR (b = '456')
>>>
>>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
>>> wrote:
>>>
>>>> Hi guys
>>>>
>>>>
>>>> I'm trying to implement some kind of NOT filter in Hbase, but don't
>> know
>>>> if possible, I'm playing with FilterIfMissing and FilterList.Operator
>> but
>>>> without luck.
>>>>
>>>>
>>>> I know how to return rows not having a specific column, but I mean
>>>> something like returning rows NOT fullfilling a condition, where
>>> condition
>>>> could be not only a SingleColumnValueFilter but a combined condition
>> with
>>>> FilterList. In SQL would be something like this for example
>>>>
>>>>
>>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
>>>>
>>>>
>>>> Thanks in advance,
>>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Carl M <so...@hotmail.com>.
Sorry for not being clear enough.
Maybe i misunderstand your suggestion, but what I've done in implement my own filter, wrapping a FilterList
public class NotFilterList extends Filter {
private FilterList filter;
public NotFilterList(final List<Filter> rowFilters) {
this.filter = new FilterList(rowFilters);
}
public void reset() throws IOException {
this.filter.reset();
}
public boolean filterRowKey(byte[] rowKey, int offset, int length) throws IOException {
return this.filter.filterRowKey(rowKey, offset, length);
}
public boolean filterAllRemaining() throws IOException {
return this.filter.filterAllRemaining();
}
public ReturnCode filterKeyValue(Cell v) throws IOException {
ReturnCode code = this.filter.filterKeyValue(v);
if (code == ReturnCode.INCLUDE)
code = ReturnCode.SKIP;
else if (code == ReturnCode.INCLUDE_AND_NEXT_COL)
code = ReturnCode.NEXT_COL;
else if (code == ReturnCode.NEXT_ROW)
code = ReturnCode.INCLUDE_AND_NEXT_COL;
else if (code == ReturnCode.SKIP) {
code = ReturnCode.INCLUDE;
}
return code;
}
public Cell transformCell(Cell v) throws IOException {
return this.filter.transformCell(v);
}
public KeyValue transform(KeyValue currentKV) throws IOException {
return this.filter.transform(currentKV);
}
public void filterRowCells(List<Cell> kvs) throws IOException {
this.filter.filterRowCells(kvs);
}
public boolean hasFilterRow() {
return this.filter.hasFilterRow();
}
public boolean filterRow() throws IOException {
return this.filter.filterRow();
}
public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException {
return this.filter.getNextKeyHint(currentKV);
}
public Cell getNextCellHint(Cell currentKV) throws IOException {
return this.filter.getNextCellHint(currentKV);
}
public boolean isFamilyEssential(byte[] name) throws IOException {
return this.filter.isFamilyEssential(name);
}
public byte[] toByteArray() throws IOException {
return this.filter.toByteArray();
}
public static NotFilterList parseFrom(final byte [] pbBytes) throws DeserializationException {
FilterList filterList = FilterList.parseFrom(pbBytes);
return new NotFilterList(filterList.getFilters());
}
boolean areSerializedFieldsEqual(Filter other) {
return this.filter.areSerializedFieldsEqual(other);
}
}
What I mean is that filterKeyValue in the way i have it now, return the right results but only the fields that were not originally skipped.
So for example, if i have two rows each one with two fields
Row 1
Name: Bill
Surname: Gates
Row 2
Name: Steve
Surname: Jobs
And I want to query for Rows that doesn't have Name 'Bill'
NOT (Name='Bill')
What I get as result from Hbase with this NotFilter is
Row 2
Surname: Jobs
I suppose it's related to the cell "Name: Steve" skipped in the first place (before reversing the ReturnCode).
Best,
________________________________
De: Ted Yu <yu...@gmail.com>
Enviado: lunes, 02 de enero de 2017 06:31 p.m.
Para: user@hbase.apache.org
Asunto: Re: Is it possible to implement a NOT filter in Hbase?
bq. the cell/value that was originally skip is not return
Can you be a bit more specific (with a concrete example) : the skipped cell
would not be returned (as indicated by the ReturnCode).
Thanks
On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
> Hi Ted,
>
>
> I tried your suggestion, unfortunately it doesn't work as expected. I
> don't fully understand FilterList, but if a cell value was skip and I
> reverse the ReturnCode, i get the right row but the cell/value that was
> originally skip is not return.
>
> I also tried reversing only filterRow() method of FilterList, but I got
> the same behaviour (the original cell/value missing).
>
>
> Best,
>
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
> not interested in any version of the current Cell.
>
> Cheers
>
> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
>
> > Thanks Ted! Great idea replacing the value in filterKeyValue. Although
> I'm
> > not quite sure looking at FilterList code if only INCLUDE/SKIP should be
> > replaced, and which should be the correct replacement for
> > INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> > implement DeMorgan's law but I think it would be harder.
> >
> >
> > Best,
> >
> > ________________________________
> > De: Ted Yu <yu...@gmail.com>
> > Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> > Para: user@hbase.apache.org
> > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >
> > You can try negating the ReturnCode from filterKeyValue() (at the root of
> > FilterList):
> >
> > abstract public ReturnCode filterKeyValue(final Cell v) throws
> > IOException;
> >
> > INCLUDE -> SKIP
> >
> > SKIP -> INCLUDE
> >
> > Alternatively, you can use De Morgan's law to transfer the condition:
> >
> > NOT (a = '123' AND b = '456') becomes
> >
> > (NOT a = '123') OR (b = '456')
> >
> > On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
> wrote:
> >
> > > Hi guys
> > >
> > >
> > > I'm trying to implement some kind of NOT filter in Hbase, but don't
> know
> > > if possible, I'm playing with FilterIfMissing and FilterList.Operator
> but
> > > without luck.
> > >
> > >
> > > I know how to return rows not having a specific column, but I mean
> > > something like returning rows NOT fullfilling a condition, where
> > condition
> > > could be not only a SingleColumnValueFilter but a combined condition
> with
> > > FilterList. In SQL would be something like this for example
> > >
> > >
> > > SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> > >
> > >
> > > Thanks in advance,
> > >
> >
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
bq. the cell/value that was originally skip is not return
Can you be a bit more specific (with a concrete example) : the skipped cell
would not be returned (as indicated by the ReturnCode).
Thanks
On Mon, Jan 2, 2017 at 1:06 PM, Carl M <so...@hotmail.com> wrote:
> Hi Ted,
>
>
> I tried your suggestion, unfortunately it doesn't work as expected. I
> don't fully understand FilterList, but if a cell value was skip and I
> reverse the ReturnCode, i get the right row but the cell/value that was
> originally skip is not return.
>
> I also tried reversing only filterRow() method of FilterList, but I got
> the same behaviour (the original cell/value missing).
>
>
> Best,
>
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
> not interested in any version of the current Cell.
>
> Cheers
>
> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
>
> > Thanks Ted! Great idea replacing the value in filterKeyValue. Although
> I'm
> > not quite sure looking at FilterList code if only INCLUDE/SKIP should be
> > replaced, and which should be the correct replacement for
> > INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> > implement DeMorgan's law but I think it would be harder.
> >
> >
> > Best,
> >
> > ________________________________
> > De: Ted Yu <yu...@gmail.com>
> > Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> > Para: user@hbase.apache.org
> > Asunto: Re: Is it possible to implement a NOT filter in Hbase?
> >
> > You can try negating the ReturnCode from filterKeyValue() (at the root of
> > FilterList):
> >
> > abstract public ReturnCode filterKeyValue(final Cell v) throws
> > IOException;
> >
> > INCLUDE -> SKIP
> >
> > SKIP -> INCLUDE
> >
> > Alternatively, you can use De Morgan's law to transfer the condition:
> >
> > NOT (a = '123' AND b = '456') becomes
> >
> > (NOT a = '123') OR (b = '456')
> >
> > On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com>
> wrote:
> >
> > > Hi guys
> > >
> > >
> > > I'm trying to implement some kind of NOT filter in Hbase, but don't
> know
> > > if possible, I'm playing with FilterIfMissing and FilterList.Operator
> but
> > > without luck.
> > >
> > >
> > > I know how to return rows not having a specific column, but I mean
> > > something like returning rows NOT fullfilling a condition, where
> > condition
> > > could be not only a SingleColumnValueFilter but a combined condition
> with
> > > FilterList. In SQL would be something like this for example
> > >
> > >
> > > SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> > >
> > >
> > > Thanks in advance,
> > >
> >
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Carl M <so...@hotmail.com>.
Hi Ted,
I tried your suggestion, unfortunately it doesn't work as expected. I don't fully understand FilterList, but if a cell value was skip and I reverse the ReturnCode, i get the right row but the cell/value that was originally skip is not return.
I also tried reversing only filterRow() method of FilterList, but I got the same behaviour (the original cell/value missing).
Best,
________________________________
De: Ted Yu <yu...@gmail.com>
Enviado: viernes, 30 de diciembre de 2016 12:56 p.m.
Para: user@hbase.apache.org
Asunto: Re: Is it possible to implement a NOT filter in Hbase?
I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
not interested in any version of the current Cell.
Cheers
On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
> Thanks Ted! Great idea replacing the value in filterKeyValue. Although I'm
> not quite sure looking at FilterList code if only INCLUDE/SKIP should be
> replaced, and which should be the correct replacement for
> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> implement DeMorgan's law but I think it would be harder.
>
>
> Best,
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> You can try negating the ReturnCode from filterKeyValue() (at the root of
> FilterList):
>
> abstract public ReturnCode filterKeyValue(final Cell v) throws
> IOException;
>
> INCLUDE -> SKIP
>
> SKIP -> INCLUDE
>
> Alternatively, you can use De Morgan's law to transfer the condition:
>
> NOT (a = '123' AND b = '456') becomes
>
> (NOT a = '123') OR (b = '456')
>
> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com> wrote:
>
> > Hi guys
> >
> >
> > I'm trying to implement some kind of NOT filter in Hbase, but don't know
> > if possible, I'm playing with FilterIfMissing and FilterList.Operator but
> > without luck.
> >
> >
> > I know how to return rows not having a specific column, but I mean
> > something like returning rows NOT fullfilling a condition, where
> condition
> > could be not only a SingleColumnValueFilter but a combined condition with
> > FilterList. In SQL would be something like this for example
> >
> >
> > SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> >
> >
> > Thanks in advance,
> >
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : you're
not interested in any version of the current Cell.
Cheers
On Fri, Dec 30, 2016 at 4:53 AM, Carl M <so...@hotmail.com> wrote:
> Thanks Ted! Great idea replacing the value in filterKeyValue. Although I'm
> not quite sure looking at FilterList code if only INCLUDE/SKIP should be
> replaced, and which should be the correct replacement for
> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to
> implement DeMorgan's law but I think it would be harder.
>
>
> Best,
>
> ________________________________
> De: Ted Yu <yu...@gmail.com>
> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
> Para: user@hbase.apache.org
> Asunto: Re: Is it possible to implement a NOT filter in Hbase?
>
> You can try negating the ReturnCode from filterKeyValue() (at the root of
> FilterList):
>
> abstract public ReturnCode filterKeyValue(final Cell v) throws
> IOException;
>
> INCLUDE -> SKIP
>
> SKIP -> INCLUDE
>
> Alternatively, you can use De Morgan's law to transfer the condition:
>
> NOT (a = '123' AND b = '456') becomes
>
> (NOT a = '123') OR (b = '456')
>
> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com> wrote:
>
> > Hi guys
> >
> >
> > I'm trying to implement some kind of NOT filter in Hbase, but don't know
> > if possible, I'm playing with FilterIfMissing and FilterList.Operator but
> > without luck.
> >
> >
> > I know how to return rows not having a specific column, but I mean
> > something like returning rows NOT fullfilling a condition, where
> condition
> > could be not only a SingleColumnValueFilter but a combined condition with
> > FilterList. In SQL would be something like this for example
> >
> >
> > SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
> >
> >
> > Thanks in advance,
> >
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Carl M <so...@hotmail.com>.
Thanks Ted! Great idea replacing the value in filterKeyValue. Although I'm not quite sure looking at FilterList code if only INCLUDE/SKIP should be replaced, and which should be the correct replacement for INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to implement DeMorgan's law but I think it would be harder.
Best,
________________________________
De: Ted Yu <yu...@gmail.com>
Enviado: jueves, 29 de diciembre de 2016 06:10 p.m.
Para: user@hbase.apache.org
Asunto: Re: Is it possible to implement a NOT filter in Hbase?
You can try negating the ReturnCode from filterKeyValue() (at the root of
FilterList):
abstract public ReturnCode filterKeyValue(final Cell v) throws
IOException;
INCLUDE -> SKIP
SKIP -> INCLUDE
Alternatively, you can use De Morgan's law to transfer the condition:
NOT (a = '123' AND b = '456') becomes
(NOT a = '123') OR (b = '456')
On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com> wrote:
> Hi guys
>
>
> I'm trying to implement some kind of NOT filter in Hbase, but don't know
> if possible, I'm playing with FilterIfMissing and FilterList.Operator but
> without luck.
>
>
> I know how to return rows not having a specific column, but I mean
> something like returning rows NOT fullfilling a condition, where condition
> could be not only a SingleColumnValueFilter but a combined condition with
> FilterList. In SQL would be something like this for example
>
>
> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
>
>
> Thanks in advance,
>
Re: Is it possible to implement a NOT filter in Hbase?
Posted by Ted Yu <yu...@gmail.com>.
You can try negating the ReturnCode from filterKeyValue() (at the root of
FilterList):
abstract public ReturnCode filterKeyValue(final Cell v) throws
IOException;
INCLUDE -> SKIP
SKIP -> INCLUDE
Alternatively, you can use De Morgan's law to transfer the condition:
NOT (a = '123' AND b = '456') becomes
(NOT a = '123') OR (b = '456')
On Thu, Dec 29, 2016 at 12:56 PM, Carl M <so...@hotmail.com> wrote:
> Hi guys
>
>
> I'm trying to implement some kind of NOT filter in Hbase, but don't know
> if possible, I'm playing with FilterIfMissing and FilterList.Operator but
> without luck.
>
>
> I know how to return rows not having a specific column, but I mean
> something like returning rows NOT fullfilling a condition, where condition
> could be not only a SingleColumnValueFilter but a combined condition with
> FilterList. In SQL would be something like this for example
>
>
> SELECT * FROM table WHERE NOT (a = '123' AND b = '456');
>
>
> Thanks in advance,
>