You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org> on 2010/07/08 22:26:49 UTC
[jira] Created: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Entire Row Deletes not stored in Row+Col Bloom
----------------------------------------------
Key: HBASE-2823
URL: https://issues.apache.org/jira/browse/HBASE-2823
Project: HBase
Issue Type: Bug
Components: filters, regionserver
Affects Versions: 0.21.0
Reporter: Nicolas Spiegelberg
Fix For: 0.21.0
If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
2. keep delete information in a separate storage element.
#1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2823:
-------------------------
Attachment: HBASE-2823.patch
Uploading v2 patch from review.hbase.org, the patch I'm about to apply
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
> Attachments: HBASE-2823.patch, HBASE-2823.patch
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray reassigned HBASE-2823:
------------------------------------
Assignee: Alexander Georgiev
Added Alexander as a contributor and assigning to him.
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.21.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.21.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893758#action_12893758 ]
HBase Review Board commented on HBASE-2823:
-------------------------------------------
Message from: "Nicolas" <ns...@facebook.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/426/#review577
-----------------------------------------------------------
Ship it!
looks good
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
<http://review.cloudera.org/r/426/#comment2220>
inverted logic not necessary here. it might be more straightforward to say:
if(type == ROWCOL) {
// comments + special stuff
} else {
// normal stuff.
}
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
<http://review.cloudera.org/r/426/#comment2219>
small typo: s/and/an/
- Nicolas
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886461#action_12886461 ]
Jonathan Gray commented on HBASE-2823:
--------------------------------------
I think I like #1. Seems like the most minimally invasive way to do this. There's virtually no impact on memory consumption, hfile size/complexity, etc... The cost will be doing two bloom checks, however bloom operations are constant time so I suspect the impact to be negligible.
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.21.0
> Reporter: Nicolas Spiegelberg
> Fix For: 0.21.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893752#action_12893752 ]
HBase Review Board commented on HBASE-2823:
-------------------------------------------
Message from: "Alexander Georgiev" <es...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/426/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
When a Delete Row is issued on a row with row+col bloom filter, some of the columns might not be deleted. Since a Delete Row is just Delete Family applied to all columns, if a file doesn't contain the column we are searching for it might end up unaffected. In order to ensure the file will be included, the row together with row+col are added in the bloom. Then shouldSeek() checks both row and row+col if the bloom is row+col (BloomType.ROWCOL). That adds additional false positives, which are taken into account with dividing the error rate the user requires by two.
This addresses bug HBASE-2823.
http://issues.apache.org/jira/browse/HBASE-2823
Diffs
-----
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 979864
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 979864
Diff: http://review.cloudera.org/r/426/diff
Testing
-------
Added new test that checks this in TestHRegion.java.
Dumped the contents of the StoreFile in order to ensure that the bloom filter has row as a value when using ROWCOL blooms.
Thanks,
Alexander
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886459#action_12886459 ]
ryan rawson commented on HBASE-2823:
------------------------------------
Deleting an entire row is just entering a delete family in to every
family, so it would make sense that these KeyValues would appear in
both the row and row+col blooms if they exist.
On Thu, Jul 8, 2010 at 1:26 PM, Nicolas Spiegelberg (JIRA)
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.21.0
> Reporter: Nicolas Spiegelberg
> Fix For: 0.21.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "Alexander Georgiev (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Georgiev updated HBASE-2823:
--------------------------------------
Attachment: HBASE-2823.patch
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
> Attachments: HBASE-2823.patch
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2823:
-------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Release Note: (was: Just a minor fix)
Resolution: Fixed
Committed. Thanks for the patch Alexander Georgiev.
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
> Attachments: HBASE-2823.patch, HBASE-2823.patch
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895023#action_12895023 ]
HBase Review Board commented on HBASE-2823:
-------------------------------------------
Message from: "Alexander Georgiev" <es...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/426/
-----------------------------------------------------------
(Updated 2010-08-03 13:04:02.245493)
Review request for hbase.
Changes
-------
Applied corrections suggested by Nicolas.
Summary
-------
When a Delete Row is issued on a row with row+col bloom filter, some of the columns might not be deleted. Since a Delete Row is just Delete Family applied to all columns, if a file doesn't contain the column we are searching for it might end up unaffected. In order to ensure the file will be included, the row together with row+col are added in the bloom. Then shouldSeek() checks both row and row+col if the bloom is row+col (BloomType.ROWCOL). That adds additional false positives, which are taken into account with dividing the error rate the user requires by two.
This addresses bug HBASE-2823.
http://issues.apache.org/jira/browse/HBASE-2823
Diffs (updated)
-----
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 979864
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 979864
Diff: http://review.cloudera.org/r/426/diff
Testing
-------
Added new test that checks this in TestHRegion.java.
Dumped the contents of the StoreFile in order to ensure that the bloom filter has row as a value when using ROWCOL blooms.
Thanks,
Alexander
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
> Attachments: HBASE-2823.patch
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886463#action_12886463 ]
Jonathan Gray commented on HBASE-2823:
--------------------------------------
This issue applies to DeleteFamily instances. (like ryan says, we have no delete row)
The issue is that a DeleteFamily applies to all columns. In a row+col bloom, you may not include a file which includes a DeleteFamily because that file doesn't contain any instances of the column you are looking for. To ensure that file would be included, we should include a key into the row+col bloom of just the row, and when checking the blooms, we should check each for row+col and for row. Note, this only applies to row+col, and in this case it is _only_ the DeleteFamilys that should add an additional key to the bloom.
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.21.0
> Reporter: Nicolas Spiegelberg
> Fix For: 0.21.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2823) Entire Row Deletes not stored in
Row+Col Bloom
Posted by "Alexander Georgiev (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Georgiev updated HBASE-2823:
--------------------------------------
Status: Patch Available (was: Open)
Release Note: Just a minor fix
Tags: hbase
When a Delete Row is issued on a row with row+col bloom filter, some of the columns might not be deleted. Since a Delete Row is just Delete Family applied to all columns, if a file doesn't contain the column we are searching for it might end up unaffected. In order to ensure the file will be included, the row together with row+col are added in the bloom. Then shouldSeek() checks both row and row+col if the bloom is row+col (BloomType.ROWCOL). That adds additional false positives, which are taken into account with dividing the error rate the user requires by two.
Added new test that checks this in TestHRegion.java.
Dumped the contents of the StoreFile in order to ensure that the bloom filter has row as a value when using ROWCOL blooms.
> Entire Row Deletes not stored in Row+Col Bloom
> ----------------------------------------------
>
> Key: HBASE-2823
> URL: https://issues.apache.org/jira/browse/HBASE-2823
> Project: HBase
> Issue Type: Bug
> Components: filters, regionserver
> Affects Versions: 0.90.0
> Reporter: Nicolas Spiegelberg
> Assignee: Alexander Georgiev
> Fix For: 0.90.0
>
>
> If the user issues a Row Delete on an family with Row+Col blooms, that information is not currently detected by shouldSeek(). Possible known solutions are:
> 1. adding Row as Bloom Filter Key on Row Delete, shouldSeek() should do both a Row & Row+Col query for Row+Col filters.
> 2. keep delete information in a separate storage element.
> #1 seems like the best solution, but need to investigate further and fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.