You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Yoom Nguyen <yo...@misoccer.us> on 2017/03/19 21:46:17 UTC
Multiple match's Filter is NOT WORKING
I am having problems getting correct match's results from searching through multiple HBASE's tables.
Is there a bug in HBASE's filter? Wondering if anyone know HBASE \ Java well and willing to help out or for a fee.
Is this a bug with HBASE?
HBASE Release 0.98.8 - 11/18/2014
Would someone help me out with the problem or confirmation whether HBASE is the right tool or not.
Here are the details of the problem:
First I am reading some data from one table and storing in a vector or arrayList: Is there a better way or different approach to achieve the results than what I am heading? I am stuck trying to get the correct results with the following approach.
//First, I am reading some data from one table and storing in a vector or arrayList:
for (Result r : rs) {
for (KeyValue kv : r.raw()) {
if (new String(kv.getFamily()).equals("name")) {
temp = new String(kv.getValue());
x.addElement(temp);
}
}
}
//Then, I want to search a different table based on the values of this vector.
//I used filters to do this: (I tried BinaryPrefixComparator and BinaryComparator as well)
FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
for (int c = 0; c < x.size(); c++) {
System.out.println(x.get(c).toString());
filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("name"), null,
CompareOp.EQUAL, new SubstringComparator( x.get(c).toString() )));
}
//I should get 3 results back, however I only get one result back, the first entry in the database.
What doesn't make sense is that when I hardcode the value that I am looking for into my code, I will get all 3 results back.
I thought there might be some issue with converting the bytes to String and then back to bytes, but that would not explain how it was able to bring back the first result. For some reason, it is stopping at the first match and doesn't continue to find the other 2 rows that contain matching data. If I hardcode it i get the results:
x.addElement("abc123");
filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("mpnum"), null, CompareOp.EQUAL, new SubstringComparator( x.get(0).toString() )));
edit: Here is the contents of the tables:
TABLE1: ROW COLUMN+CELL
0 column=gpnum:, timestamp=1481300288449, value=def123
0 column=mpnum:, timestamp=1481300273355, value=abc123
0 column=price:, timestamp=1481300255337, value=85.0
1 column=gpnum:, timestamp=1481301599999, value=def2244
1 column=mpnum:, timestamp=1481301582336, value=011511607
1 column=price:, timestamp=1481301673886, value=0.76
TABLE2
ROW COLUMN+CELL
0 column=brand:, timestamp=1481300227283, value=x
0 column=mpnum:, timestamp=1481300212289, value=abc123
0 column=price:, timestamp=1481300110950, value=50.0
1 column=mpnum:, timestamp=1481301806687 , value=011511607
1 column=price:, timestamp=1481301777345 , value=1.81
13 column=webtype:, timestamp=1483507543878, value=US
3 column=avail:, timestamp=1481306538360, value=avail
3 column=brand:, timestamp=1481306538360, value=brand
3 column=descr:, timestamp=1481306538360, value=description
3 column=dist:, timestamp=1481306538360, value=distributor
3 column=mpnum:, timestamp=1481306538360, value=pnum
3 column=price:, timestamp=1481306538360, value=price
3 column=url:, timestamp=1481306538360, value=url
3 column=webtype:, timestamp=1481306538360, value=webtype
4 column=avail:, timestamp=1481306538374, value=4
4 column=brand:, timestamp=1481306538374, value=x
4 column=descr:, timestamp=1481306538374, value=description
4 column=dist:, timestamp=1481306538374, value=x
4 column=mpnum:, timestamp=1482117383212 , value=011511607
4 column=price:, timestamp=1481306538374 , value=34.51
4 column=url:, timestamp=1481306538374, value=x:q!
4 column=webtype:, timestamp=1481306538374, value=US
5 column=avail:, timestamp=1481306538378, value=
5 column=brand:, timestamp=1481306538378, value=name
5 column=descr:, timestamp=1481306538378, value=x
5 column=dist:, timestamp=1481306538378, value=x
5 column=mpnum:, timestamp=1482117392043 , value=011511607
5 column=price:, timestamp=1481306538378 , value=321.412
5 column=url:, timestamp=1481306538378, value=x.com
THIRD TABLE (to store result matches)
0 column=brand:, timestamp=1481301813849, value=name
0 column=cprice:, timestamp=1481301813849, value=1.81
0 column=gpnum:, timestamp=1481301813849, value=def2244
0 column=gprice:, timestamp=1481301813849, value=0.76
0 column=mpnum:, timestamp=1481301813849, value=011511607
**should be three matches those that are in bold above but only brings back one match
Your help is much appreciated. Thank You Yoom
Re: Multiple match's Filter is NOT WORKING
Posted by Josh Elser <el...@apache.org>.
Yoom,
Please leave conversation on the mailing list.
I am suggesting that you create a standalone application which writes data
in your format and then tries to read it using your filter logic that
doesn't work. It should be something that we can run. This would let us
actually see what isn't working.
On Mar 20, 2017 20:45, "Yoom Nguyen" <yo...@misoccer.us> wrote:
Josh,
Thank you for replying. I am not quite sure how to do what you are
suggesting. I will have figure it out.
Sorry, I am not trying to threatening, we like HBASE that is why we choose
it as a technology for our project.
I am run into a road block and not while sure how to proceed from here
without getting data back correctly from
HBASE. We could not move forward due to the problem we are having.
Can you help? or suggest someone might interest and can help. I will send
you give a dinner gift certificate to where you want to go for two if you
could look into my problem and provide some suggestion.
Thanks,
Yoom
----- Original Message -----
From: "Josh Elser" <el...@apache.org>
To: dev@hbase.apache.org
Sent: Monday, March 20, 2017 6:29:41 PM
Subject: Re: Multiple match's Filter is NOT WORKING
If you could encapsulate what you're trying to do into a
unit-test/standalone class, it would likely be much more approachable
for anyone to reproduce and help debug your issues.
You've provided a lot of information, but very little of it is helpful
in actually figuring out why what you've done doesn't work. Most of the
time, the devil is in the details which your code snippets wouldn't
capture in the first place.
Please do remember that we're all volunteers. We want to try to help,
but you'll get a much better response with the carrot (effort from your
side to show what doesn't work) than the stick (threatening to "give up
using HBase").
Yoom Nguyen wrote:
> Would anyone know if this is the right forum to ask this type of
question? If not, would you suggest where I should try to ask this type of
question before I am giving up using HBASE all together.
> I am stuck for a while now and not able to get any advice from anyone.
>
> Thanks in advance,
> Yoom
>
> ----- Original Message -----
>
> From: "Yoom Nguyen"<yo...@misoccer.us>
> To: dev@hbase.apache.org
> Sent: Sunday, March 19, 2017 5:46:17 PM
> Subject: Multiple match's Filter is NOT WORKING
>
>
> I am having problems getting correct match's results from searching
through multiple HBASE's tables.
> Is there a bug in HBASE's filter? Wondering if anyone know HBASE \ Java
well and willing to help out or for a fee.
> Is this a bug with HBASE?
>
> HBASE Release 0.98.8 - 11/18/2014
>
>
> Would someone help me out with the problem or confirmation whether HBASE
is the right tool or not.
>
> Here are the details of the problem:
> First I am reading some data from one table and storing in a vector or
arrayList: Is there a better way or different approach to achieve the
results than what I am heading? I am stuck trying to get the correct
results with the following approach.
>
> //First, I am reading some data from one table and storing in a vector or
arrayList:
>
> for (Result r : rs) {
> for (KeyValue kv : r.raw()) {
> if (new String(kv.getFamily()).equals("name")) {
> temp = new String(kv.getValue());
> x.addElement(temp);
> }
> }
> }
>
>
> //Then, I want to search a different table based on the values of this
vector.
> //I used filters to do this: (I tried BinaryPrefixComparator and
BinaryComparator as well)
>
>
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
>
> for (int c = 0; c< x.size(); c++) {
> System.out.println(x.get(c).toString());
> filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("name"),
null,
> CompareOp.EQUAL, new SubstringComparator( x.get(c).toString() )));
> }
>
> //I should get 3 results back, however I only get one result back, the
first entry in the database.
> What doesn't make sense is that when I hardcode the value that I am
looking for into my code, I will get all 3 results back.
>
> I thought there might be some issue with converting the bytes to String
and then back to bytes, but that would not explain how it was able to bring
back the first result. For some reason, it is stopping at the first match
and doesn't continue to find the other 2 rows that contain matching data.
If I hardcode it i get the results:
>
>
> x.addElement("abc123");
> filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("mpnum"),
null, CompareOp.EQUAL, new SubstringComparator( x.get(0).toString() )));
>
>
> edit: Here is the contents of the tables:
>
> TABLE1: ROW COLUMN+CELL
> 0 column=gpnum:, timestamp=1481300288449, value=def123
> 0 column=mpnum:, timestamp=1481300273355, value=abc123
> 0 column=price:, timestamp=1481300255337, value=85.0
> 1 column=gpnum:, timestamp=1481301599999, value=def2244
> 1 column=mpnum:, timestamp=1481301582336, value=011511607
> 1 column=price:, timestamp=1481301673886, value=0.76
>
> TABLE2
>
> ROW COLUMN+CELL
> 0 column=brand:, timestamp=1481300227283, value=x
> 0 column=mpnum:, timestamp=1481300212289, value=abc123
> 0 column=price:, timestamp=1481300110950, value=50.0
> 1 column=mpnum:, timestamp=1481301806687 , value=011511607
> 1 column=price:, timestamp=1481301777345 , value=1.81
> 13 column=webtype:, timestamp=1483507543878, value=US
> 3 column=avail:, timestamp=1481306538360, value=avail
> 3 column=brand:, timestamp=1481306538360, value=brand
> 3 column=descr:, timestamp=1481306538360, value=description
> 3 column=dist:, timestamp=1481306538360, value=distributor
> 3 column=mpnum:, timestamp=1481306538360, value=pnum
> 3 column=price:, timestamp=1481306538360, value=price
> 3 column=url:, timestamp=1481306538360, value=url
> 3 column=webtype:, timestamp=1481306538360, value=webtype
> 4 column=avail:, timestamp=1481306538374, value=4
> 4 column=brand:, timestamp=1481306538374, value=x
> 4 column=descr:, timestamp=1481306538374, value=description
> 4 column=dist:, timestamp=1481306538374, value=x
> 4 column=mpnum:, timestamp=1482117383212 , value=011511607
> 4 column=price:, timestamp=1481306538374 , value=34.51
> 4 column=url:, timestamp=1481306538374, value=x:q!
> 4 column=webtype:, timestamp=1481306538374, value=US
> 5 column=avail:, timestamp=1481306538378, value=
> 5 column=brand:, timestamp=1481306538378, value=name
> 5 column=descr:, timestamp=1481306538378, value=x
> 5 column=dist:, timestamp=1481306538378, value=x
> 5 column=mpnum:, timestamp=1482117392043 , value=011511607
> 5 column=price:, timestamp=1481306538378 , value=321.412
> 5 column=url:, timestamp=1481306538378, value=x.com
>
> THIRD TABLE (to store result matches)
>
> 0 column=brand:, timestamp=1481301813849, value=name
> 0 column=cprice:, timestamp=1481301813849, value=1.81
> 0 column=gpnum:, timestamp=1481301813849, value=def2244
> 0 column=gprice:, timestamp=1481301813849, value=0.76
> 0 column=mpnum:, timestamp=1481301813849, value=011511607
>
> **should be three matches those that are in bold above but only brings
back one match
>
> Your help is much appreciated. Thank You Yoom
>
Re: Multiple match's Filter is NOT WORKING
Posted by Josh Elser <el...@apache.org>.
If you could encapsulate what you're trying to do into a
unit-test/standalone class, it would likely be much more approachable
for anyone to reproduce and help debug your issues.
You've provided a lot of information, but very little of it is helpful
in actually figuring out why what you've done doesn't work. Most of the
time, the devil is in the details which your code snippets wouldn't
capture in the first place.
Please do remember that we're all volunteers. We want to try to help,
but you'll get a much better response with the carrot (effort from your
side to show what doesn't work) than the stick (threatening to "give up
using HBase").
Yoom Nguyen wrote:
> Would anyone know if this is the right forum to ask this type of question? If not, would you suggest where I should try to ask this type of question before I am giving up using HBASE all together.
> I am stuck for a while now and not able to get any advice from anyone.
>
> Thanks in advance,
> Yoom
>
> ----- Original Message -----
>
> From: "Yoom Nguyen"<yo...@misoccer.us>
> To: dev@hbase.apache.org
> Sent: Sunday, March 19, 2017 5:46:17 PM
> Subject: Multiple match's Filter is NOT WORKING
>
>
> I am having problems getting correct match's results from searching through multiple HBASE's tables.
> Is there a bug in HBASE's filter? Wondering if anyone know HBASE \ Java well and willing to help out or for a fee.
> Is this a bug with HBASE?
>
> HBASE Release 0.98.8 - 11/18/2014
>
>
> Would someone help me out with the problem or confirmation whether HBASE is the right tool or not.
>
> Here are the details of the problem:
> First I am reading some data from one table and storing in a vector or arrayList: Is there a better way or different approach to achieve the results than what I am heading? I am stuck trying to get the correct results with the following approach.
>
> //First, I am reading some data from one table and storing in a vector or arrayList:
>
> for (Result r : rs) {
> for (KeyValue kv : r.raw()) {
> if (new String(kv.getFamily()).equals("name")) {
> temp = new String(kv.getValue());
> x.addElement(temp);
> }
> }
> }
>
>
> //Then, I want to search a different table based on the values of this vector.
> //I used filters to do this: (I tried BinaryPrefixComparator and BinaryComparator as well)
>
>
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
>
> for (int c = 0; c< x.size(); c++) {
> System.out.println(x.get(c).toString());
> filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("name"), null,
> CompareOp.EQUAL, new SubstringComparator( x.get(c).toString() )));
> }
>
> //I should get 3 results back, however I only get one result back, the first entry in the database.
> What doesn't make sense is that when I hardcode the value that I am looking for into my code, I will get all 3 results back.
>
> I thought there might be some issue with converting the bytes to String and then back to bytes, but that would not explain how it was able to bring back the first result. For some reason, it is stopping at the first match and doesn't continue to find the other 2 rows that contain matching data. If I hardcode it i get the results:
>
>
> x.addElement("abc123");
> filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("mpnum"), null, CompareOp.EQUAL, new SubstringComparator( x.get(0).toString() )));
>
>
> edit: Here is the contents of the tables:
>
> TABLE1: ROW COLUMN+CELL
> 0 column=gpnum:, timestamp=1481300288449, value=def123
> 0 column=mpnum:, timestamp=1481300273355, value=abc123
> 0 column=price:, timestamp=1481300255337, value=85.0
> 1 column=gpnum:, timestamp=1481301599999, value=def2244
> 1 column=mpnum:, timestamp=1481301582336, value=011511607
> 1 column=price:, timestamp=1481301673886, value=0.76
>
> TABLE2
>
> ROW COLUMN+CELL
> 0 column=brand:, timestamp=1481300227283, value=x
> 0 column=mpnum:, timestamp=1481300212289, value=abc123
> 0 column=price:, timestamp=1481300110950, value=50.0
> 1 column=mpnum:, timestamp=1481301806687 , value=011511607
> 1 column=price:, timestamp=1481301777345 , value=1.81
> 13 column=webtype:, timestamp=1483507543878, value=US
> 3 column=avail:, timestamp=1481306538360, value=avail
> 3 column=brand:, timestamp=1481306538360, value=brand
> 3 column=descr:, timestamp=1481306538360, value=description
> 3 column=dist:, timestamp=1481306538360, value=distributor
> 3 column=mpnum:, timestamp=1481306538360, value=pnum
> 3 column=price:, timestamp=1481306538360, value=price
> 3 column=url:, timestamp=1481306538360, value=url
> 3 column=webtype:, timestamp=1481306538360, value=webtype
> 4 column=avail:, timestamp=1481306538374, value=4
> 4 column=brand:, timestamp=1481306538374, value=x
> 4 column=descr:, timestamp=1481306538374, value=description
> 4 column=dist:, timestamp=1481306538374, value=x
> 4 column=mpnum:, timestamp=1482117383212 , value=011511607
> 4 column=price:, timestamp=1481306538374 , value=34.51
> 4 column=url:, timestamp=1481306538374, value=x:q!
> 4 column=webtype:, timestamp=1481306538374, value=US
> 5 column=avail:, timestamp=1481306538378, value=
> 5 column=brand:, timestamp=1481306538378, value=name
> 5 column=descr:, timestamp=1481306538378, value=x
> 5 column=dist:, timestamp=1481306538378, value=x
> 5 column=mpnum:, timestamp=1482117392043 , value=011511607
> 5 column=price:, timestamp=1481306538378 , value=321.412
> 5 column=url:, timestamp=1481306538378, value=x.com
>
> THIRD TABLE (to store result matches)
>
> 0 column=brand:, timestamp=1481301813849, value=name
> 0 column=cprice:, timestamp=1481301813849, value=1.81
> 0 column=gpnum:, timestamp=1481301813849, value=def2244
> 0 column=gprice:, timestamp=1481301813849, value=0.76
> 0 column=mpnum:, timestamp=1481301813849, value=011511607
>
> **should be three matches those that are in bold above but only brings back one match
>
> Your help is much appreciated. Thank You Yoom
>
Re: Multiple match's Filter is NOT WORKING
Posted by Yoom Nguyen <yo...@misoccer.us>.
Would anyone know if this is the right forum to ask this type of question? If not, would you suggest where I should try to ask this type of question before I am giving up using HBASE all together.
I am stuck for a while now and not able to get any advice from anyone.
Thanks in advance,
Yoom
----- Original Message -----
From: "Yoom Nguyen" <yo...@misoccer.us>
To: dev@hbase.apache.org
Sent: Sunday, March 19, 2017 5:46:17 PM
Subject: Multiple match's Filter is NOT WORKING
I am having problems getting correct match's results from searching through multiple HBASE's tables.
Is there a bug in HBASE's filter? Wondering if anyone know HBASE \ Java well and willing to help out or for a fee.
Is this a bug with HBASE?
HBASE Release 0.98.8 - 11/18/2014
Would someone help me out with the problem or confirmation whether HBASE is the right tool or not.
Here are the details of the problem:
First I am reading some data from one table and storing in a vector or arrayList: Is there a better way or different approach to achieve the results than what I am heading? I am stuck trying to get the correct results with the following approach.
//First, I am reading some data from one table and storing in a vector or arrayList:
for (Result r : rs) {
for (KeyValue kv : r.raw()) {
if (new String(kv.getFamily()).equals("name")) {
temp = new String(kv.getValue());
x.addElement(temp);
}
}
}
//Then, I want to search a different table based on the values of this vector.
//I used filters to do this: (I tried BinaryPrefixComparator and BinaryComparator as well)
FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
for (int c = 0; c < x.size(); c++) {
System.out.println(x.get(c).toString());
filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("name"), null,
CompareOp.EQUAL, new SubstringComparator( x.get(c).toString() )));
}
//I should get 3 results back, however I only get one result back, the first entry in the database.
What doesn't make sense is that when I hardcode the value that I am looking for into my code, I will get all 3 results back.
I thought there might be some issue with converting the bytes to String and then back to bytes, but that would not explain how it was able to bring back the first result. For some reason, it is stopping at the first match and doesn't continue to find the other 2 rows that contain matching data. If I hardcode it i get the results:
x.addElement("abc123");
filterList.addFilter(new SingleColumnValueFilter(Bytes.toBytes("mpnum"), null, CompareOp.EQUAL, new SubstringComparator( x.get(0).toString() )));
edit: Here is the contents of the tables:
TABLE1: ROW COLUMN+CELL
0 column=gpnum:, timestamp=1481300288449, value=def123
0 column=mpnum:, timestamp=1481300273355, value=abc123
0 column=price:, timestamp=1481300255337, value=85.0
1 column=gpnum:, timestamp=1481301599999, value=def2244
1 column=mpnum:, timestamp=1481301582336, value=011511607
1 column=price:, timestamp=1481301673886, value=0.76
TABLE2
ROW COLUMN+CELL
0 column=brand:, timestamp=1481300227283, value=x
0 column=mpnum:, timestamp=1481300212289, value=abc123
0 column=price:, timestamp=1481300110950, value=50.0
1 column=mpnum:, timestamp=1481301806687 , value=011511607
1 column=price:, timestamp=1481301777345 , value=1.81
13 column=webtype:, timestamp=1483507543878, value=US
3 column=avail:, timestamp=1481306538360, value=avail
3 column=brand:, timestamp=1481306538360, value=brand
3 column=descr:, timestamp=1481306538360, value=description
3 column=dist:, timestamp=1481306538360, value=distributor
3 column=mpnum:, timestamp=1481306538360, value=pnum
3 column=price:, timestamp=1481306538360, value=price
3 column=url:, timestamp=1481306538360, value=url
3 column=webtype:, timestamp=1481306538360, value=webtype
4 column=avail:, timestamp=1481306538374, value=4
4 column=brand:, timestamp=1481306538374, value=x
4 column=descr:, timestamp=1481306538374, value=description
4 column=dist:, timestamp=1481306538374, value=x
4 column=mpnum:, timestamp=1482117383212 , value=011511607
4 column=price:, timestamp=1481306538374 , value=34.51
4 column=url:, timestamp=1481306538374, value=x:q!
4 column=webtype:, timestamp=1481306538374, value=US
5 column=avail:, timestamp=1481306538378, value=
5 column=brand:, timestamp=1481306538378, value=name
5 column=descr:, timestamp=1481306538378, value=x
5 column=dist:, timestamp=1481306538378, value=x
5 column=mpnum:, timestamp=1482117392043 , value=011511607
5 column=price:, timestamp=1481306538378 , value=321.412
5 column=url:, timestamp=1481306538378, value=x.com
THIRD TABLE (to store result matches)
0 column=brand:, timestamp=1481301813849, value=name
0 column=cprice:, timestamp=1481301813849, value=1.81
0 column=gpnum:, timestamp=1481301813849, value=def2244
0 column=gprice:, timestamp=1481301813849, value=0.76
0 column=mpnum:, timestamp=1481301813849, value=011511607
**should be three matches those that are in bold above but only brings back one match
Your help is much appreciated. Thank You Yoom