You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Bryan Duxbury (JIRA)" <ji...@apache.org> on 2008/03/17 21:16:25 UTC
[jira] Created: (HBASE-524) Problems with getFull
Problems with getFull
---------------------
Key: HBASE-524
URL: https://issues.apache.org/jira/browse/HBASE-524
Project: Hadoop HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.1.0, 0.2.0
Reporter: Bryan Duxbury
Assignee: Bryan Duxbury
Priority: Critical
There are some issues with the implementation of getFull in HStore.
* If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
* Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HBASE-524:
--------------------------------
Fix Version/s: 0.1.0
Status: Patch Available (was: Open)
Please review.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0
>
> Attachments: 524-0.1.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579647#action_12579647 ]
Bryan Duxbury commented on HBASE-524:
-------------------------------------
The second problem above was invalid, it actually was scanning in the right order.
The second problem was a real issue, and in fact there was another problem: if the deletes weren't precisely matched to cells by timestamp, they would be ignored, resulting in an inaccurate view of the row.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HBASE-524:
--------------------------------
Attachment: 524-trunk.patch
Here's the same patch cut for trunk. Unit tests pass locally.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0
>
> Attachments: 524-0.1.patch, 524-trunk.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-524) Problems with getFull
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580003#action_12580003 ]
stack commented on HBASE-524:
-----------------------------
+1
Reviewed both patches. Looks good. In future, suggest using defines/constants for stuff used in 2 or more places such as this:
{code}
+ batchUpdate.put(COLUMNS[0], "olderValue".getBytes());
{code}
Later you test "olderValue" made it into the table.
I ran the 0.1 junit suite and all passed.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0, 0.2.0
>
> Attachments: 524-0.1.patch, 524-trunk.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HBASE-524:
--------------------------------
Fix Version/s: 0.2.0
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0, 0.2.0
>
> Attachments: 524-0.1.patch, 524-trunk.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HBASE-524:
--------------------------------
Attachment: 524-0.1.patch
This patch introduces a new test case to make prove this stuff is actually broken, and then fixes to HStore to make it work as intended. All tests pass.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0
>
> Attachments: 524-0.1.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-524) Problems with getFull
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HBASE-524:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this to 0.1 and trunk.
> Problems with getFull
> ---------------------
>
> Key: HBASE-524
> URL: https://issues.apache.org/jira/browse/HBASE-524
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.0, 0.2.0
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Critical
> Fix For: 0.1.0, 0.2.0
>
> Attachments: 524-0.1.patch, 524-trunk.patch
>
>
> There are some issues with the implementation of getFull in HStore.
> * If the loop encounters a deleted cell, it stops iterating. This correctly handles deletes, but then accidentally masks away any cells of different qualifiers that would come afterward
> * Since the mapfiles are search oldest to newest, and the results map is only updated when there isn't already a value in the results map for for the cell we're currently looking at, older values actually take precedence over newer ones. This may be fixed by simply reversing the order of mapfiles traversed to newest to oldest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.