You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Peter Haidinyak <ph...@local.com> on 2011/01/25 00:20:29 UTC
Scan Question
If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
Thanks
-Pete
RE: Scan Question
Posted by Peter Haidinyak <ph...@local.com>.
Thanks, that fixed my problem. I knew it had to be something I did or didn't do. I had just copied the code from one of my working apps but its tables didn't have any columns with empty values.
Thanks again.
-Pete
-----Original Message-----
From: Ryan Rawson [mailto:ryanobjc@gmail.com]
Sent: Monday, January 24, 2011 4:24 PM
To: user@hbase.apache.org
Subject: Re: Scan Question
if you look at the code, in the top of SingleColumnValueFilter, the
javadoc says:
"To prevent the entire row from being emitted if the column is not
found on a row, use setFilterIfMissing. Otherwise, if the column is
found, the entire row will be emitted only if the value passes. If the
value fails, the row will be filtered out."
The default value of filterIfMissing is FALSE, meaning if a row does
NOT contain the column you are looking for, it is included. If it
does contain the column you are looking for AND the comparison is
true, include else exclude.
that is probably the source of the confusion...
setFilterIfMissing(true). API wise it will be a little annoying,
sorry about that.
-ryan
On Mon, Jan 24, 2011 at 4:15 PM, Peter Haidinyak <ph...@local.com> wrote:
> In a scan I setup a filter as follows...
>
> final FilterList filterList = new FilterList();
> final String botFilterString = getFilter(BOT_VALUE);
> if (botFilterString != null)
> {
> m_logger.info("Bot Value Filter: " + botFilterString);
> final SingleColumnValueFilter botFilter = new SingleColumnValueFilter(TRAFFIC_FAMILY_BYTES,
> BOT_VALUE.getColumnAsBytes(),
> CompareFilter.CompareOp.EQUAL,
> botFilterString.getBytes());
> filterList.addFilter(botFilter);
> }
>
> I then add the filterList to the Scan object. The problem I am having is if I setup a filter that I know should not return any values I still get values returned. What is also interesting is that some of the filters work as expected and some don't. I've had several sets of eyes look at the code and couldn't see any obvious problems.
>
>
> Thanks
>
> -Pete
>
> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Monday, January 24, 2011 3:58 PM
> To: user@hbase.apache.org
> Subject: Re: Scan Question
>
> CompareFilter is just an abstract base of a series of other filters
> that compare specific components, what exactly are you having a
> problem with?
>
>
>
> On Mon, Jan 24, 2011 at 3:53 PM, Peter Haidinyak <ph...@local.com> wrote:
>> That's the way I thought it should work. When I setup a filter with data that I know shouldn't return any data I still get rows back.
>>
>> -----Original Message-----
>> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
>> Sent: Monday, January 24, 2011 3:32 PM
>> To: user@hbase.apache.org
>> Subject: Re: Scan Question
>>
>> no, if you provide a list of columns in your Scan query and a
>> particular row does not actually contain _that specific column_, then
>> the filter does not see anything and nothing for that row is returned.
>>
>> -ryan
>>
>> On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
>>> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>>
>>>
>>> Thanks
>>>
>>> -Pete
>>>
>>
>
Re: Scan Question
Posted by Ryan Rawson <ry...@gmail.com>.
if you look at the code, in the top of SingleColumnValueFilter, the
javadoc says:
"To prevent the entire row from being emitted if the column is not
found on a row, use setFilterIfMissing. Otherwise, if the column is
found, the entire row will be emitted only if the value passes. If the
value fails, the row will be filtered out."
The default value of filterIfMissing is FALSE, meaning if a row does
NOT contain the column you are looking for, it is included. If it
does contain the column you are looking for AND the comparison is
true, include else exclude.
that is probably the source of the confusion...
setFilterIfMissing(true). API wise it will be a little annoying,
sorry about that.
-ryan
On Mon, Jan 24, 2011 at 4:15 PM, Peter Haidinyak <ph...@local.com> wrote:
> In a scan I setup a filter as follows...
>
> final FilterList filterList = new FilterList();
> final String botFilterString = getFilter(BOT_VALUE);
> if (botFilterString != null)
> {
> m_logger.info("Bot Value Filter: " + botFilterString);
> final SingleColumnValueFilter botFilter = new SingleColumnValueFilter(TRAFFIC_FAMILY_BYTES,
> BOT_VALUE.getColumnAsBytes(),
> CompareFilter.CompareOp.EQUAL,
> botFilterString.getBytes());
> filterList.addFilter(botFilter);
> }
>
> I then add the filterList to the Scan object. The problem I am having is if I setup a filter that I know should not return any values I still get values returned. What is also interesting is that some of the filters work as expected and some don't. I've had several sets of eyes look at the code and couldn't see any obvious problems.
>
>
> Thanks
>
> -Pete
>
> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Monday, January 24, 2011 3:58 PM
> To: user@hbase.apache.org
> Subject: Re: Scan Question
>
> CompareFilter is just an abstract base of a series of other filters
> that compare specific components, what exactly are you having a
> problem with?
>
>
>
> On Mon, Jan 24, 2011 at 3:53 PM, Peter Haidinyak <ph...@local.com> wrote:
>> That's the way I thought it should work. When I setup a filter with data that I know shouldn't return any data I still get rows back.
>>
>> -----Original Message-----
>> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
>> Sent: Monday, January 24, 2011 3:32 PM
>> To: user@hbase.apache.org
>> Subject: Re: Scan Question
>>
>> no, if you provide a list of columns in your Scan query and a
>> particular row does not actually contain _that specific column_, then
>> the filter does not see anything and nothing for that row is returned.
>>
>> -ryan
>>
>> On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
>>> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>>
>>>
>>> Thanks
>>>
>>> -Pete
>>>
>>
>
RE: Scan Question
Posted by Peter Haidinyak <ph...@local.com>.
In a scan I setup a filter as follows...
final FilterList filterList = new FilterList();
final String botFilterString = getFilter(BOT_VALUE);
if (botFilterString != null)
{
m_logger.info("Bot Value Filter: " + botFilterString);
final SingleColumnValueFilter botFilter = new SingleColumnValueFilter(TRAFFIC_FAMILY_BYTES,
BOT_VALUE.getColumnAsBytes(),
CompareFilter.CompareOp.EQUAL,
botFilterString.getBytes());
filterList.addFilter(botFilter);
}
I then add the filterList to the Scan object. The problem I am having is if I setup a filter that I know should not return any values I still get values returned. What is also interesting is that some of the filters work as expected and some don't. I've had several sets of eyes look at the code and couldn't see any obvious problems.
Thanks
-Pete
-----Original Message-----
From: Ryan Rawson [mailto:ryanobjc@gmail.com]
Sent: Monday, January 24, 2011 3:58 PM
To: user@hbase.apache.org
Subject: Re: Scan Question
CompareFilter is just an abstract base of a series of other filters
that compare specific components, what exactly are you having a
problem with?
On Mon, Jan 24, 2011 at 3:53 PM, Peter Haidinyak <ph...@local.com> wrote:
> That's the way I thought it should work. When I setup a filter with data that I know shouldn't return any data I still get rows back.
>
> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Monday, January 24, 2011 3:32 PM
> To: user@hbase.apache.org
> Subject: Re: Scan Question
>
> no, if you provide a list of columns in your Scan query and a
> particular row does not actually contain _that specific column_, then
> the filter does not see anything and nothing for that row is returned.
>
> -ryan
>
> On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
>> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>
>>
>> Thanks
>>
>> -Pete
>>
>
Re: Scan Question
Posted by Ryan Rawson <ry...@gmail.com>.
CompareFilter is just an abstract base of a series of other filters
that compare specific components, what exactly are you having a
problem with?
On Mon, Jan 24, 2011 at 3:53 PM, Peter Haidinyak <ph...@local.com> wrote:
> That's the way I thought it should work. When I setup a filter with data that I know shouldn't return any data I still get rows back.
>
> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Monday, January 24, 2011 3:32 PM
> To: user@hbase.apache.org
> Subject: Re: Scan Question
>
> no, if you provide a list of columns in your Scan query and a
> particular row does not actually contain _that specific column_, then
> the filter does not see anything and nothing for that row is returned.
>
> -ryan
>
> On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
>> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>
>>
>> Thanks
>>
>> -Pete
>>
>
RE: Scan Question
Posted by Peter Haidinyak <ph...@local.com>.
That's the way I thought it should work. When I setup a filter with data that I know shouldn't return any data I still get rows back.
-----Original Message-----
From: Ryan Rawson [mailto:ryanobjc@gmail.com]
Sent: Monday, January 24, 2011 3:32 PM
To: user@hbase.apache.org
Subject: Re: Scan Question
no, if you provide a list of columns in your Scan query and a
particular row does not actually contain _that specific column_, then
the filter does not see anything and nothing for that row is returned.
-ryan
On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>
> Thanks
>
> -Pete
>
Re: Scan Question
Posted by Ryan Rawson <ry...@gmail.com>.
no, if you provide a list of columns in your Scan query and a
particular row does not actually contain _that specific column_, then
the filter does not see anything and nothing for that row is returned.
-ryan
On Mon, Jan 24, 2011 at 3:20 PM, Peter Haidinyak <ph...@local.com> wrote:
> If I have a table where some of the columns might not have values in each row and I do a scan with a CompareFilter.CompareOp.EQUAL type filter on one of those columns will the scan bring back rows where there is no value in the column I am comparing on?
>
> Thanks
>
> -Pete
>