You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Amitanand Aiyer (JIRA)" <ji...@apache.org> on 2011/09/07 20:11:10 UTC
[jira] [Created] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Ensure that Scanners that read from the storefiles respect MVCC
---------------------------------------------------------------
Key: HBASE-4345
URL: https://issues.apache.org/jira/browse/HBASE-4345
Project: HBase
Issue Type: Sub-task
Reporter: Amitanand Aiyer
Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
"newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Attachment: patch-3
To enforce the read-write consistency mechanism, we need to ignore all key-values that have a memstoreTS greater than the read point for the get/scan operation.
Filters provide an excellent mechanism to ignore values based on any desired condition. We implement the RWCC mechanism for the key-values read from the disk by including a filter for the scan object that ignores "newer" reads.
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Status: Patch Available (was: Open)
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099249#comment-13099249 ]
stack commented on HBASE-4345:
------------------------------
Interesting take, enforcing we only read at the read point or before by doing the functionality as a filter that is always used. Intellectually I'd say that this should be core and not be done via a filter but if filter gets the job done, lets go for it. Nice.
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113408#comment-13113408 ]
Amitanand Aiyer commented on HBASE-4345:
----------------------------------------
I have modified the ScanQueryMatcher to take care of ignoring "newer" KV's. If done by the
Scan query matcher, it is also going to disregard the deletes so it will help us with other
issues.
4344-v8 includes this change.
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099290#comment-13099290 ]
Ted Yu commented on HBASE-4345:
-------------------------------
Minor comment:
{code}
+ @Override
+ public boolean hasFilterRow() {
+ if (existingFilter == null)
+ return false;
+ else
+ return existingFilter.hasFilterRow();
+ }
{code}
else is not needed above.
What's the plan for all the sub-tasks of HBASE-2856 ?
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer resolved HBASE-4345.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
fix included in 2856-v6.txt
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.94.0, 0.89.20100924
>
> Attachments: 4345-v2.txt, patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Status: Open (was: Patch Available)
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112693#comment-13112693 ]
Amitanand Aiyer commented on HBASE-4345:
----------------------------------------
@stack: I'm going to redo this patch. I gave walk through of this code last week, internally, and one of the
comments I got was that doing the filter approach is *not* going to ignore delete-markers that occur after
the read point. i.e. they will take effect, even though they should not.
I've been in austin for a recruiting trip, so don't have the patch ready. Will try to get it and update asap.
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Status: Open (was: Patch Available)
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: 4345-v2.txt, patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Fix Version/s: (was: 0.94.0)
0.89.20100924
Assignee: Amitanand Aiyer
Status: Patch Available (was: Open)
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104134#comment-13104134 ]
stack commented on HBASE-4345:
------------------------------
@Amit You have any revisions on this patch since you wrote it? Should we commit?
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4345) Ensure that Scanners that read from
the storefiles respect MVCC
Posted by "Amitanand Aiyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-4345:
-----------------------------------
Attachment: 4345-v2.txt
> Ensure that Scanners that read from the storefiles respect MVCC
> ---------------------------------------------------------------
>
> Key: HBASE-4345
> URL: https://issues.apache.org/jira/browse/HBASE-4345
> Project: HBase
> Issue Type: Sub-task
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.89.20100924
>
> Attachments: 4345-v2.txt, patch-3
>
>
> Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
> information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
> "newer" writes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira