You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2012/06/11 23:37:43 UTC
[jira] [Created] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Jean-Daniel Cryans created HBASE-6200:
-----------------------------------------
Summary: KeyComparator.compareWithoutRow can be wrong when families have the same prefix
Key: HBASE-6200
URL: https://issues.apache.org/jira/browse/HBASE-6200
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0, 0.92.1, 0.90.6
Reporter: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
As reported by magma on IRC (AKA Desert Rose on the ML), Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393509#comment-13393509 ]
Lars Hofhansl commented on HBASE-6200:
--------------------------------------
The problem is that families are not compared at all (so in the case above '' is less than 'a').
Actually, that is correct as far as the StoreScanners, etc, are concerned, as there need not be an explicit ordering between KVs in different CFs - they are stored in different stores. So the cycles comparing families can be spared.
Of course, when we want to sort KVs from multiple stores (as in this case a ResultSet) this comes back to bite us.
It seems we need an extra comparator that includes families, for use in cases where KVs from multiple stores are compared.
I forgot why we decided not remove all the comparators from the KeyValue class, but in this case there could be a comparator in hbase-server which ignores families and one in common (or client in the future) that includes them.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: PerformanceTestCase-6200-94.patch
This is how I did this test:
1. Test comparing between 'famia:qualia' with 'famib:qualia'. Run for 1000000*2 times. Calculate the time consumed(Using System.currentTimeMillis() to get current time).
2. Test comparing between 'fami:qualia' with 'fami:qualib'. Run for 1,000,000*2 times. Calculate the time consumed.
3. Repeats 1~2 for 20 times. Accumulate the total consumed time at step 1 and step 2, and then calculate for the average time.
4. Repeats 1~3 for 3 groups.
Please find the attached file "PerformanceTestCase-6200-94.patch" for details.
And this is the test results in my machine(Machine info: 64-bit Suse, CPU: 16 * 1.6GHz, Memory: 32GB):
[With patch 6200]
{noformat}
Group-1:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2425, average -> 121
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 3282, average -> 164
Group-2:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2421, average -> 121
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 3279, average -> 163
Group-3:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2417, average -> 120
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 3279, average -> 163
{noformat}
[Without patch 6200]
{noformat}
Group-1:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2015, average -> 100
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 2191, average -> 109
Group-12:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2014, average -> 100
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 2186, average -> 109
Group-13:
Compare {famia:qualia} with {famib:qualia} used time(totally) -> 2012, average -> 100
Compare {fami:qualia} with {fami:qualib} used time(totally) -> 2186, average -> 109
{noformat}
Plz share your comments:)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-94-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403271#comment-13403271 ]
Hudson commented on HBASE-6200:
-------------------------------
Integrated in HBase-0.94-security #38 (See [https://builds.apache.org/job/HBase-0.94-security/38/])
HBASE-6200 KeyComparator.compareWithoutRow can be wrong when families have the same prefix (Jieshan) (Revision 1354293)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6200:
---------------------------------
Status: Patch Available (was: Open)
Thanks Jieshan!
Looks good off-hand (haven't looked too closely, though). Let's get a test-run in.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0, 0.92.1, 0.90.6
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393515#comment-13393515 ]
Lars Hofhansl commented on HBASE-6200:
--------------------------------------
Meh... Looking again at the code. Families are compared. Just ignore me :)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-trunk.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: PerformanceTest-trunk.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-trunk.patch
A patch of separately comparison for CF and Qualiifer.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400248#comment-13400248 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
bq.Patch looks good. I'm not sure I'm clear what commonPrefix is... how its used. For example, what is going on here:
bq.We move from commonPrefix+4 to commonPrefix+6. Is this commonPrefix w/i the row only? Or is it row+? It seems its row? Its not clear from the javadoc nor as I walk through.
Sorry for the confusion. I will add more comments to explain this:). I will refresh the patches today.
bq.Probably we should use variable of same name instead of commonPrefix.
Ya...I should use "commonLength":)
bq.Also with this change, since KVs are compared extremely frequently we should performance test this.
Lars, do you have any suggestions on this?:)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393626#comment-13393626 ]
stack commented on HBASE-6200:
------------------------------
Seems like a 'hole' since elsewhere in KV we compare columns by looking at their column family first, then the column qualifiers: e.g. KeyValue#compareColumns.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398390#comment-13398390 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
Hadoop QA is not functioning due to svn version.
Running tests locally.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6200:
--------------------------------------
Description:
As reported by Sergio Esteves on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
was:
As reported by magma on IRC (AKA Desert Rose on the ML), Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Sergio Esteves on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401829#comment-13401829 ]
Hudson commented on HBASE-6200:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #70 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/70/])
HBASE-6200 KeyComparator.compareWithoutRow can be wrong when families have the same prefix (Jieshan) (Revision 1354290)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: 6200-0.92.txt
Patch for 0.92
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400265#comment-13400265 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533238/HBASE-6200-trunk-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2245//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2245//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: 6200-90.patch
Patch for 90 based on V2. Local tests are running, I will tell the results once it finish..
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399363#comment-13399363 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532999/HBASE-6200-94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2221//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400205#comment-13400205 ]
Lars Hofhansl commented on HBASE-6200:
--------------------------------------
Also with this change, since KVs are compared extremely frequently we should performance test this.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: 6200-trunk-v2.patch
Re-attaching patch v2.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401508#comment-13401508 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
I ran the two tests listed above and they passed.
@Jieshan:
Please prepare patches for 0.94, 0.92 and 0.90 branches based on patch v4.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400529#comment-13400529 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
@Lars:
What other tests do you think should be performed ?
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: 6200-trunk-v3.patch
I tried my best to decrease the calculate times in this version of patch. And I changed the test case to test comparison directly（Do 1,000,000 compares each time）.
With the same steps:
1. Test comparing between 'famia:qualia' with 'famib:qualia'. Run for 1000000 times. Calculate the time consumed(Using System.currentTimeMillis() to get current time).
2. Test comparing between 'fami:qualia' with 'fami:qualib'. Run for 1,000,000 times. Calculate the time consumed.
3. Repeats 1~2 for 20 times. Accumulate the total consumed time at step 1 and step 2, and then calculate for the average time.
Test code:
{noformat}
for (int loop = 0; loop < 20; loop++) {
long start = System.currentTimeMillis();
for (int i = 0; i < 1000000; i++) {
compareIgnoringPrefix(c, 0, kvf_a, kvf_b);
}
long end = System.currentTimeMillis();
long useTimeA = end - start;
start = end;
for (int i = 0; i < 1000000; i++) {
compareIgnoringPrefix(c, 0, kvq_a, kvq_b);
}
end = System.currentTimeMillis();
long useTimeB = end - start;
totalTimeA += useTimeA;
totalTimeB += useTimeB;
}
private void compareIgnoringPrefix(KeyValue.KeyComparator c, int common, KeyValue less,
KeyValue greater) {
int cmp = c.compareIgnoringPrefix(common, less.getBuffer(), less.getOffset()
+ KeyValue.ROW_OFFSET, less.getKeyLength(), greater.getBuffer(),
greater.getOffset() + KeyValue.ROW_OFFSET, greater.getKeyLength());
}
{noformat}
And this is the new result:
[without patch 6200]
{noformat}
Compare {famia:qualia} with {famib:qualia}, run for 1,000,000 times. used time -> 50
Compare {fami:qualia} with {fami:qualib}, run for 1,000,000 times. used time -> 58
{noformat}
[with patch 6200]
{noformat}
Compare {famia:qualia} with {famib:qualia}, run for 1,000,000 times. used time -> 56
Compare {fami:qualia} with {fami:qualib}, run for 1,000,000 times. used time -> 64
{noformat}
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6200) KeyComparator.compareWithoutRow can be
wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl closed HBASE-6200.
--------------------------------
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400276#comment-13400276 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
@Jieshan:
Let's start with your test plan.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-trunk-v2.patch
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398466#comment-13398466 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
@Jieshan:
Please provide patches for other branches after addressing review comments.
Thanks
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-trunk-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-90.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-92.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6200:
--------------------------------------
Description:
As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
was:
As reported by Sergio Esteves on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
Changing the user's name again at his request.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Status: Patch Available (was: Open)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0, 0.92.1, 0.90.6
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: PerformanceTestCase-6200-94.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-94.patch
HBASE-6200-92.patch
HBASE-6200-90.patch
Patches for all other branches.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: 6200-trunk-v4.txt
Patch v4 reorders some assignments so that variables are calculated immediately before their usage.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-trunk-v2.patch
Added more comments for the test case.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6200)
KeyComparator.compareWithoutRow can be wrong when families have the same
prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399202#comment-13399202 ]
Zhihong Ted Yu edited comment on HBASE-6200 at 6/22/12 8:24 AM:
----------------------------------------------------------------
All Apache build servers are offline.
Please run test suite in all branches.
Please list the failing tests, is any, individually so that people can help verify.
was (Author: zhihyu@ebaysf.com):
All Apache build servers are offline.
Please list the failing tests, is any, individually so that people can help verify.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-92-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399701#comment-13399701 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
>From earlier in the patch:
{code}
+ int commonLength = ROW_LENGTH_SIZE + FAMILY_LENGTH_SIZE + rowlength;
{code}
Probably we should use variable of same name instead of commonPrefix.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401943#comment-13401943 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
All tests passed for 90.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6200:
---------------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533286/PerformanceTestCase-6200-94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2249//console
This message is automatically generated.)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398505#comment-13398505 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
Thank you for your review, Ted. I will update the patch and make the patches for other branches tommorow.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean reassigned HBASE-6200:
-----------------------------------
Assignee: Jieshan Bean
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400698#comment-13400698 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
@Stack:
Do you have further comments on the latest patches ?
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Status: Open (was: Patch Available)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0, 0.92.1, 0.90.6
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399401#comment-13399401 ]
stack commented on HBASE-6200:
------------------------------
Patch looks good. I'm not sure I'm clear what commonPrefix is... how its used. For example, what is going on here:
{code}
+ // 'fami:qf1' < 'fami:qf2' With commonPrefix + 4 bytes prefix.
+ assertKVLessWithoutRow(c, commonPrefix + 4, kv0_1, kv0_2);
+ // 'fami:qf1' < 'fami:qf2' With commonPrefix + 6 bytes prefix.
+ assertKVLessWithoutRow(c, commonPrefix + 6, kv0_1, kv0_2);
{code}
We move from commonPrefix+4 to commonPrefix+6. Is this commonPrefix w/i the row only? Or is it row+? It seems its row? Its not clear from the javadoc nor as I walk through.
This stuff is hard to follow at the best of times so while it looks like this is fixed with this patch -- thanks for doing it -- any chance of a bit more explaination either w/ comments or improved javadoc.
Otherwise, patch looks great. Tests are nice.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401951#comment-13401951 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
Integrated to 0.90 branch as well.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398177#comment-13398177 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
bq.They should be compared separately, but then the comparator would need to look up the actual family length and call Bytes.compareTo twice.
Agree with this idea. I think call Bytes.compareTo twice is not a problem. Anyway put them together and do the comparison is incorrect.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan resolved HBASE-6200.
-------------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Resolving as committed to all branches.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: PerformanceTest-trunk.patch
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Status: Open (was: Patch Available)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0, 0.92.1, 0.90.6
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400252#comment-13400252 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533236/HBASE-6200-trunk-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2243//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2243//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400462#comment-13400462 ]
Lars Hofhansl commented on HBASE-6200:
--------------------------------------
So if I read that right 1.000.000 KV compares take 100ms without the patch and 121ms with the patch. I think that would be within the noise of any real operation.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401205#comment-13401205 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
The above approach should work.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401807#comment-13401807 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
Patch for 0.94 is same as the patch for trunk.
I ran tests for 0.92 and they passed.
Integrated to trunk, 0.94 and 0.92
Will integrate to 0.90 once the patch is ready.
Thanks for the patch, Jieshan.
Thanks for the review, Stack and Lars.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400254#comment-13400254 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
For performance test, how about using LoadTestTool / ycsb (same parameters) on builds with and without this patch ?
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401167#comment-13401167 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
The extra Bytes.compareTo() call is behind the difference of performance.
If we can enhance the following:
{code}
public static int compareTo(byte[] buffer1, int offset1, int length1,
byte[] buffer2, int offset2, int length2) {
{code}
so that it accepts four pairs of (offset, length) tuple, both column family and column qualifier can be compared in one trip using unsafe comparer.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-trunk-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399446#comment-13399446 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533061/6200-trunk-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2223//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2223//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2223//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2223//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-90-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-trunk-v2.patch
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401094#comment-13401094 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
bq.So if I read that right 1.000.000 KV compares take 100ms without the patch and 121ms with the patch.
1,000,000 times called the below method(Will compare 2 times in each call), to be exact:
{noformat}
private void assertKVLessWithoutRow(KeyValue.KeyComparator c, int common, KeyValue less,
KeyValue greater) {
int cmp = c.compareIgnoringPrefix(common, less.getBuffer(), less.getOffset()
+ KeyValue.ROW_OFFSET, less.getKeyLength(), greater.getBuffer(),
greater.getOffset() + KeyValue.ROW_OFFSET, greater.getKeyLength());
assertTrue(cmp < 0);
cmp = c.compareIgnoringPrefix(common, greater.getBuffer(), greater.getOffset()
+ KeyValue.ROW_OFFSET, greater.getKeyLength(), less.getBuffer(),
less.getOffset() + KeyValue.ROW_OFFSET, less.getKeyLength());
assertTrue(cmp > 0);
}
{noformat}
Thanks, Lars & Ted.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: (was: 6200-trunk-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400264#comment-13400264 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533237/6200-trunk-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2244//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2244//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400268#comment-13400268 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533251/HBASE-6200-94-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2246//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: HBASE-6200-94-v2.patch
HBASE-6200-92-v2.patch
HBASE-6200-90-v2.patch
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401841#comment-13401841 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
Thank you Ted. I will upload the patch for 90 immediately(I planed to make all the patches after review:))..
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400256#comment-13400256 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
I agree with this idea. We should create one table with several families which have the same prefix. And test the load/scan performance.
Before that, I can write one test case to test that method only(w/ and w/o this patch). And then decide whether do more tests. Is that ok?
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401124#comment-13401124 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
1,000,000 * 2 comparison, 100ms Vs 121ms. In real environment, this is a magnified value.
Since it's a bug, we should discuss whether this decline is tolerable. If not, how to optimize and fix this problem?
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398460#comment-13398460 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
Patch looks great.
Minor comments:
{code}
+ * validate that assumption here.
+ *
{code}
There is a whitespace at the end of second line above.
{code}
+ int lqualifierlength = llength - TIMESTAMP_TYPE_SIZE
+ - (lfamilyoffset - loffset) - lfamilylength;
{code}
lfamilyoffset - loffset is actually commonLength. Replace it with commonLength makes code easier to read.
Same with the computation of rqualifierlength
There're 5 new white spaces in TestKeyValue
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401861#comment-13401861 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
Sorry.. It was made based on V4 not V2..
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393517#comment-13393517 ]
Lars Hofhansl commented on HBASE-6200:
--------------------------------------
So family and qualifier are compared together. So above compared 'f1' with 'fa', which is wrong.
They should be compared separately, but then the comparator would need to look up the actual family length and call Bytes.compareTo twice.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399202#comment-13399202 ]
Zhihong Ted Yu commented on HBASE-6200:
---------------------------------------
All Apache build servers are offline.
Please list the failing tests, is any, individually so that people can help verify.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401454#comment-13401454 ]
stack commented on HBASE-6200:
------------------------------
Thanks for taking time on perf Jieshan. +1 on commit (It looks like you have enough tests).
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401195#comment-13401195 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
Ya...You reminded me. I have one idea to optimize this:
1. if (left family length != right family length). Only compare column family is enough.
2. if (left family length == right family length). we can put them together to compare.
So no mattter which case, only one comparison will happen.
I will test it right now.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400323#comment-13400323 ]
Jieshan Bean commented on HBASE-6200:
-------------------------------------
@Anoop:
Thanks for your carefully review.
Only the createLastOnRow methods using Type.Minimum. Likes below:
{noformat}
public static KeyValue createLastOnRow(final byte[] row) {
return new KeyValue(row, null, null, HConstants.LATEST_TIMESTAMP, Type.Minimum);
}
{noformat}
At this time, the length of CF (or Qualifier) should be 0.
Actually, we can create one KeyValue with CF as empty but Qualifier is not empty. And if we call KeyValue#getQualifier, we can get the correct value. So I think this check should be strict with both CF == 0 and Qualifier == 0.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: 6200-trunk-v2.patch
Most recent QA run encountered:
{code}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase-server: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create new native thread -> [Help 1]
[ERROR]
{code}
Attaching patch again.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401851#comment-13401851 ]
Hudson commented on HBASE-6200:
-------------------------------
Integrated in HBase-0.94 #284 (See [https://builds.apache.org/job/HBase-0.94/284/])
HBASE-6200 KeyComparator.compareWithoutRow can be wrong when families have the same prefix (Jieshan) (Revision 1354293)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-94.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-92-v2.patch, HBASE-6200-94-v2.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400443#comment-13400443 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533286/PerformanceTestCase-6200-94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2249//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400303#comment-13400303 ]
Anoop Sam John commented on HBASE-6200:
---------------------------------------
{code}
- if (lcolumnlength == 0 && ltype == Type.Minimum.getCode()) {
+ if (lfamilylength == 0 && lqualifierlength == 0
+ && ltype == Type.Minimum.getCode()) {
{code}
May be only lfamilylength ==0 check is enough. When the family is missing from KV there can not be a qualifier.
Same with
{code}
- if (rcolumnlength == 0 && rtype == Type.Minimum.getCode()) {
+ if (rfamilylength == 0 && rqualifierlength == 0
+ && rtype == Type.Minimum.getCode()) {
{code}
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6200:
----------------------------------
Attachment: 6200-0.94.txt
Patch for 0.94
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Jieshan Bean (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-6200:
--------------------------------
Attachment: (was: HBASE-6200-trunk-v2.patch)
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, HBASE-6200-90.patch, HBASE-6200-92.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, HBASE-6200-trunk.patch
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401868#comment-13401868 ]
Hudson commented on HBASE-6200:
-------------------------------
Integrated in HBase-TRUNK #3077 (See [https://builds.apache.org/job/HBase-TRUNK/3077/])
HBASE-6200 KeyComparator.compareWithoutRow can be wrong when families have the same prefix (Jieshan) (Revision 1354290)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401483#comment-13401483 ]
Hadoop QA commented on HBASE-6200:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533487/6200-trunk-v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.security.access.TestAccessController
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2260//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2260//console
This message is automatically generated.
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can
be wrong when families have the same prefix
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401927#comment-13401927 ]
Hudson commented on HBASE-6200:
-------------------------------
Integrated in HBase-0.92 #459 (See [https://builds.apache.org/job/HBase-0.92/459/])
HBASE-6200 KeyComparator.compareWithoutRow can be wrong when families have the same prefix (Jieshan) (Revision 1354294)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
> KeyComparator.compareWithoutRow can be wrong when families have the same prefix
> -------------------------------------------------------------------------------
>
> Key: HBASE-6200
> URL: https://issues.apache.org/jira/browse/HBASE-6200
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6, 0.92.1, 0.94.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jieshan Bean
> Priority: Blocker
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: 6200-0.92.txt, 6200-0.94.txt, 6200-90.patch, 6200-trunk-v2.patch, 6200-trunk-v3.patch, 6200-trunk-v4.txt
>
>
> As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird behavior when some families share the same prefix. He posted a link to his code to show how it fails, http://pastebin.com/7TBA1XGh
> Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families and qualifiers so "f:a" is said to be bigger than "f1:", which is false. Then what happens is that the KVs are returned in the right order from the RS but then doing {{Result.binarySearch}} it uses {{KeyComparator.compareWithoutRow}} which has a different sorting so the end result is undetermined.
> I added some debug and I can see that the data is returned in the right order but {{Arrays.binarySearch}} returned the wrong KV, which is then verified agains the passed family and qualifier which fails so null is returned.
> I don't know how frequent it is for users to have families with the same prefix, but those that do have that and that use those families at the same time will have big correctness issues. This is why I mark this as a blocker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira