You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Davis Abraham (JIRA)" <ji...@apache.org> on 2012/05/07 16:26:49 UTC
[jira] [Created] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Davis Abraham created HBASE-5951:
------------------------------------
Summary: Combining ColumnPrefixFilters using filterlist.
Key: HBASE-5951
URL: https://issues.apache.org/jira/browse/HBASE-5951
Project: HBase
Issue Type: Bug
Components: filters
Affects Versions: 0.92.1
Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
Reporter: Davis Abraham
FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
SingleColumnValueFilter SingleFilter1 = new
SingleColumnValueFilter(Bytes.toBytes("cf"),
Bytes.toBytes("country"), CompareOp.EQUAL,
Bytes.toBytes("USA"));
listOfFilters.addFilter(SingleFilter1);
ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
new SubstringComparator("ABC"));
ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
listOfFilters1.addFilter(CF1);
listOfFilters1.addFilter(VF1);
listOfFilters.addFilter(listOfFilters1);
ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
new SubstringComparator("ED"));
ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
listOfFilters2.addFilter(CF2);
listOfFilters2.addFilter(VF2);
listOfFilters.addFilter(listOfFilters2);
When i do a combibation of SingleFilter1 and listOfFilters1
the result is correct, same way the combination of
SingleFilter1 and listOfFilters2 is returing correct result.
But when all the three is combined im not getting any result..
Is it the problem with multiple ColumnPrefixFilter???
Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270508#comment-13270508 ]
ramkrishna.s.vasudevan commented on HBASE-5951:
-----------------------------------------------
@Davis
I have a question.
{code}
(Select * from namexxx where value like 'ABC%')
{code}
Can we have a query where the column name itself can be prefix type in normal SQL? Columnprefix filters help you to select columns that are with the needed prefix rather than how singlecolumnvaluefilters(works on Kvs) work.
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Davis Abraham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271097#comment-13271097 ]
Davis Abraham commented on HBASE-5951:
--------------------------------------
Its not possible in normal SQL... While posting the SQL query i had mentioned that it wont work in SQl...
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271508#comment-13271508 ]
Anoop Sam John commented on HBASE-5951:
---------------------------------------
Davis , this bug can be closed now?
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Davis Abraham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270354#comment-13270354 ]
Davis Abraham commented on HBASE-5951:
--------------------------------------
Thanks Anoop for the reply.
I'm trying implementing custom filters... Is there any link where i can get samples on how to create and use a Custom Filter...
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Davis Abraham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272081#comment-13272081 ]
Davis Abraham commented on HBASE-5951:
--------------------------------------
We can close the bug by concluding that Hbase cannot support multiple ColumnPrefixFilters using filterlist...
It would have been better if Hbase could support multiple ColumnPrefixValueFilter(Yet to achieve)....:-)
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270358#comment-13270358 ]
Anoop Sam John commented on HBASE-5951:
---------------------------------------
I think better you can follow the code in SingleColumnValueFilter. You need change in filterKeyValue(KeyValue keyValue) where it checks for the column name
{code}
if (!keyValue.matchingColumn(this.columnFamily, this.columnQualifier)) {
return ReturnCode.INCLUDE;
}
{code}
Here you need to do check like ColumnPrefixFilter to know whether the KV coming is of a column of user interest, CRSxxxx
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270169#comment-13270169 ]
Anoop Sam John commented on HBASE-5951:
---------------------------------------
@Davis
It would be better if you describe your schema and the query what you are trying to generate.
The case which you describe as not working involve 2 ColumnPrefixFilter with AND condition.
ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));- This will filter out all KVs with qualifier name other than nameXXXX
The othe one, ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));- will filter out all KVs with qualifier name other than CRSXXXX
So all KVs getting filtered out?
If you tell your query need, we can discuss how the same can be created using Filter.
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270325#comment-13270325 ]
Anoop Sam John commented on HBASE-5951:
---------------------------------------
I think I somewhat got the usecase of yours. But for this you can not use the filters and the filter list as you have used in the pasted sample
Filterlist1 = ColumnPrefixFilter1 (namexxx) + valueFilter1;
Filterlist2 = ColumnPrefixFilter2 (CRSxxx) + valueFilter2;
Filterlist = Filterlist1 + Filterlist2; // AND condition
Here Filterlist1 will remove all the KVs (columns) other than the namexxx named column for a given row. So only that column(s) will be there in the result which is having a qualifier name starts with "name".
[This will even remove the KVs corresponding to CRSxxx column also]
Same way Filterlist2 will allow only columns with name CRSxxx to come out. This will filter out namexxxx columns also. So in the next step of filtering as per the final Filterlist , even the namexxx columns(KVs) also gone! So that is why ultimately you will not get any thing in the output..
For your need you need to write your own Filter implementation (like the one u said ColumnPrefixValueFilter) ... U can not use the SingleColumnValueFilter also as it expects a column ( qualifier ) name but for you that name is also like not fixed. It is like CRSxxxx.....
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Davis Abraham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270359#comment-13270359 ]
Davis Abraham commented on HBASE-5951:
--------------------------------------
Thank you....
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anoop Sam John resolved HBASE-5951.
-----------------------------------
Resolution: Not A Problem
This is not a defect.. 2 ColumnPrefixFilters were used with AND condition and that is why all the KVs were getting filtered out. For the usage scenario custom filter needs to be written.
Closing after the agreement from Davis
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "Davis Abraham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270300#comment-13270300 ]
Davis Abraham commented on HBASE-5951:
--------------------------------------
I'm trying to execute a query which should work like ColumnPrefixValueFilter (Which is not available). To attain this i combined ColumnPrefixFilter with valueFilter to search for a particular value in a set of columns.
Following is a SQL equivalent query even though it wont work in sql.
(Select * from namexxx where value like 'ABC%') AND (Select * from CRSxxx where value like 'ED%')
The result i expect is the set of rows where both the above conditions are satisfied.
I tried with some sample tables where ColumnFamily is same for all columns in all rows.
>From Hbase documentations and blogs i understood the following hierarchy can be achieved using FilterList.
Filterlist1 = ColumnPrefixFilter1 + valueFilter1; (This alone will return result)
Filterlist2 = ColumnPrefixFilter2 + valueFilter2; (This alone will also return result)
Filterlist = Filterlist1 + Filterlist2; (Result is null when combined)
The result set is always null when Filterlist1 and Filterlist2 are applied together.
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5951) Combining ColumnPrefixFilters using
filterlist.
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271137#comment-13271137 ]
ramkrishna.s.vasudevan commented on HBASE-5951:
-----------------------------------------------
bq.even though it wont work in sql.
Sorry i missed it.
> Combining ColumnPrefixFilters using filterlist.
> -----------------------------------------------
>
> Key: HBASE-5951
> URL: https://issues.apache.org/jira/browse/HBASE-5951
> Project: HBase
> Issue Type: Bug
> Components: filters
> Affects Versions: 0.92.1
> Environment: Combining multiple ColumnPrefixFilters using filterlist does not return exact result.
> Reporter: Davis Abraham
>
> FilterList listOfFilters = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters1 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> FilterList listOfFilters2 = new FilterList (FilterList.Operator.MUST_PASS_ALL);
> SingleColumnValueFilter SingleFilter1 = new
> SingleColumnValueFilter(Bytes.toBytes("cf"),
> Bytes.toBytes("country"), CompareOp.EQUAL,
> Bytes.toBytes("USA"));
> listOfFilters.addFilter(SingleFilter1);
> ValueFilter VF1= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ABC"));
> ColumnPrefixFilter CF1= new ColumnPrefixFilter(Bytes.toBytes("name"));
> listOfFilters1.addFilter(CF1);
> listOfFilters1.addFilter(VF1);
> listOfFilters.addFilter(listOfFilters1);
> ValueFilter VF2= new ValueFilter (CompareOp.EQUAL,
> new SubstringComparator("ED"));
> ColumnPrefixFilter CF2= new ColumnPrefixFilter (Bytes.toBytes("CRS"));
> listOfFilters2.addFilter(CF2);
> listOfFilters2.addFilter(VF2);
> listOfFilters.addFilter(listOfFilters2);
> When i do a combibation of SingleFilter1 and listOfFilters1
> the result is correct, same way the combination of
> SingleFilter1 and listOfFilters2 is returing correct result.
> But when all the three is combined im not getting any result..
> Is it the problem with multiple ColumnPrefixFilter???
> Value "ABC" exist in name.0 and value "ED" exist in CRS.0 and it is in
> the same row under same Column Family.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira