You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jonathan Gray (JIRA)" <ji...@apache.org> on 2011/01/05 18:10:47 UTC
[jira] Created: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
-------------------------------------------------------------------------------------------------
Key: HBASE-3418
URL: https://issues.apache.org/jira/browse/HBASE-3418
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Priority: Critical
Fix For: 0.90.1, 0.92.0
Doing investigation around some observed resetting counter behavior.
An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977864#action_12977864 ]
Todd Lipcon commented on HBASE-3418:
------------------------------------
Can we put this in 0.90? seems like inaccurate counters are a bad problem!
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.1, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-3418:
---------------------------------
Attachment: HBASE-3418-v1.patch
Unit test which reproduces bad behavior and small fix which seems to work / fixes test.
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.1, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-3418:
---------------------------------
Fix Version/s: (was: 0.90.1)
0.90.0
Yeah, looks like we're doing at least one more RC.
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.0, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray resolved HBASE-3418.
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed to branch and trunk. Thanks for review Stack.
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.0, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979008#action_12979008 ]
Hudson commented on HBASE-3418:
-------------------------------
Integrated in HBase-TRUNK #1708 (See [https://hudson.apache.org/hudson/job/HBase-TRUNK/1708/])
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.0, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3418) Increment operations can break when
qualifiers are split between memstore/snapshot and storefiles
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977896#action_12977896 ]
stack commented on HBASE-3418:
------------------------------
+1 on patch.
> Increment operations can break when qualifiers are split between memstore/snapshot and storefiles
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-3418
> URL: https://issues.apache.org/jira/browse/HBASE-3418
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.90.0, 0.92.0
>
> Attachments: HBASE-3418-v1.patch
>
>
> Doing investigation around some observed resetting counter behavior.
> An optimization was added to check memstore/snapshots first and then check storefiles if not all counters were found. However it looks like this introduced a bug when columns for a given row/family in a single increment operation are spread across memstores and storefiles.
> The results from get operations on both memstores and storefiles are appended together but when processed are expected to be fully sorted. This can lead to invalid results.
> Need to sort the combined result of memstores + storefiles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.