You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Amitanand Aiyer (Created) (JIRA)" <ji...@apache.org> on 2012/02/16 20:06:59 UTC
[jira] [Created] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit
Assign different memstoreTS to different KV's in the same WALEdit
-----------------------------------------------------------------
Key: HBASE-5414
URL: https://issues.apache.org/jira/browse/HBASE-5414
Project: HBase
Issue Type: Sub-task
Reporter: Amitanand Aiyer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Amitanand Aiyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209838#comment-13209838 ]
Amitanand Aiyer commented on HBASE-5414:
----------------------------------------
There seem to be 3 approachs we can take in terms of how we split the WALEdit into different batch of mutations (with same memstoreTS).
Option (i): Current approach. Split when we see a change in KV's type (Delete --> Put, or Put --> Delete).
Not very accurate. We might combine multiple Put mutations into one mutation. But, this should be relatively insignificant.
Option (ii): Split per KV. Give a different memstoreTS to each KV.
We might be splitting a single mutation into multiple parts. But, this should not be a problem. No readers start until the replay is done.
Option (iii): Find a way to mark the exact boundary.
Will entail inserting a "dummy/sentinel" KV into the WALEdit to represent the boundary.
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234587#comment-13234587 ]
Lars Hofhansl commented on HBASE-5414:
--------------------------------------
@Amit: You want this in 0.94? Is it ready?
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Kannan Muthukkaruppan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209675#comment-13209675 ]
Kannan Muthukkaruppan commented on HBASE-5414:
----------------------------------------------
Amit:
Aren't there three pieces of the puzzle to ensure that a atomic row mutation (RowMutations) which contains a Delete & a Put, for example:
RowMutations(Delete R1.CF @ NOW, // delete the row in this CF
Put R1.CF.c1 = v1 @ NOW) // and atomically add back a column for this row in this CF
can guarantee that the Put wins over the Delete even though both happened at the same NOW timestamp.
part 1) On insert, making sure the Put gets a higher memstoreTS than the Delete even thought they have the same TS.
part 2) On recovery (that's the piece that this JIRA seems to be covering).
part 3) On flush/compactions, etc. not doing the space optimization to clear out memstoreTS to 0 even if the read point for the active scanners has moved on, unless it is a major compaction where we lose the Delete markers.
Do we not need parts #1 & #3? Or where you planning to do those as separate tasks?
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-5414:
-------------------------------
Attachment: HBASE-5414.D1749.1.patch
aaiyer requested code review of "HBASE-5414 [jira] Assign different memstoreTS to different KV's in the same WALEdit during replay".
Reviewers: JIRA
Initial diff for splitting the KV's in the same WALEdit into different
memstoreTS operations upon replay.
TBD:
Running unit tests on MR
Need to augument the test suite to reproduce the issue and fix.
<a href="https://issues.apache.org/jira/browse/HBASE-5203" title="Group atomic put/delete operation into a single WALEdit to handle region server failures."><del>HBASE-5203</del></a> combines all the different Puts/Deletes into one WALEdit. This is
required to ensure that we persist the atomic mutation in its enterity and not
in parts.
When combined into a single WALEdit, we create one big familyMap that is a combination
of all the family maps in the mutations. The KV's in this familyMap have no information
about memstoreTS (it is not yet assigned).
However, when we apply the mutations to the Memstore (if there are no failures) we end up
incrementing the memstoreTS for each operation.
This can lead to the client seeing different order of operations – depending on weather or
not there was a RS crash/restart.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D1749
AFFECTED FILES
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/3741/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209640#comment-13209640 ]
Phabricator commented on HBASE-5414:
------------------------------------
aaiyer has commented on the revision "HBASE-5414 [jira] Assign different memstoreTS to different KV's in the same WALEdit during replay".
INLINE COMMENTS
:2839 TBD: fix comments length.
REVISION DETAIL
https://reviews.facebook.net/D1749
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Amitanand Aiyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209822#comment-13209822 ]
Amitanand Aiyer commented on HBASE-5414:
----------------------------------------
My bad. we were indeed using the same memstoreTS for all the mutations in the RowMutation. So, delete would win automatically.
Created a jira/updated a patch so we can ensure that the winning update is determined based on the order in which the client adds operations to RowMutations.
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209776#comment-13209776 ]
Phabricator commented on HBASE-5414:
------------------------------------
aaiyer has commented on the revision "HBASE-5414 [jira] Assign different memstoreTS to different KV's in the same WALEdit during replay".
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:2842 Agreed.
Keeping track of the timestamp might be tricky.We could bump up the memstoreTS for every KV that we see. this will resolve the contention between Put/Put as well. The current approach does not.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:2856 Kind of .... This is to increment the memstoreTS across *different* WALEdits.
The one inside the loop is incremented when we see different types within the *same* WALEdit.
REVISION DETAIL
https://reviews.facebook.net/D1749
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209684#comment-13209684 ]
stack commented on HBASE-5414:
------------------------------
bq. ....can guarantee that the Put wins over the Delete even though both happened at the same NOW timestamp
Trying to understand... I don't get it Kannan. If they both went in together at same TS, doesn't the Delete sort before the Put so we'd see it first... so it would overshadow the contemporaneous Put?
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit
Posted by "Amitanand Aiyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-5414:
-----------------------------------
Description:
HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
required to ensure that we persist the atomic mutation in its enterity and not
in parts.
When combined into a single WALEdit, we create one big familyMap that is a combination
of all the family maps in the mutations. The KV's in this familyMap have no information
about memstoreTS (it is not yet assigned).
However, when we apply the mutations to the Memstore (if there are no failures) we end up
incrementing the memstoreTS for each operation.
This can lead to the client seeing different order of operations -- depending on weather or
not there was a RS crash/restart.
> Assign different memstoreTS to different KV's in the same WALEdit
> -----------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Amitanand Aiyer (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer reassigned HBASE-5414:
--------------------------------------
Assignee: Amitanand Aiyer
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209657#comment-13209657 ]
Phabricator commented on HBASE-5414:
------------------------------------
stack has commented on the revision "HBASE-5414 [jira] Assign different memstoreTS to different KV's in the same WALEdit during replay".
INLINE COMMENTS
:2842 Is this enough? What if the edits have same timestamp? Should we bump the memstoreTSAtStartup then too?
Is memstoreTs a good name? Should it be memstoreSeqnumber (This would be a big change and I don't expect it as part of this patch -- just raising the point)
:2856 Why this increment? Is it a just-in-case?
REVISION DETAIL
https://reviews.facebook.net/D1749
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Amitanand Aiyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209781#comment-13209781 ]
Amitanand Aiyer commented on HBASE-5414:
----------------------------------------
@Stack. Yes. That is the current behavior.
What we are trying to get is to see if we can provide a semantics where .... the client is able to specify the order in which the operations are applied. (This is already the case wrt the implementation, because we loop through the mutations in the AtomicRowMutation and apply them one by one -- in the same order in which they were applied.)
So, part 1) seems to have been automatically addressed. Except for the corner race-condition which we will need to fix, so scans/gets do not see a partial application of a AtomicRowMutation.
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Amitanand Aiyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amitanand Aiyer updated HBASE-5414:
-----------------------------------
Summary: Assign different memstoreTS to different KV's in the same WALEdit during replay (was: Assign different memstoreTS to different KV's in the same WALEdit)
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Fix For: 0.94.0
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5414) Assign different memstoreTS to
different KV's in the same WALEdit during replay
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5414:
---------------------------------
Fix Version/s: (was: 0.94.0)
0.96.0
> Assign different memstoreTS to different KV's in the same WALEdit during replay
> -------------------------------------------------------------------------------
>
> Key: HBASE-5414
> URL: https://issues.apache.org/jira/browse/HBASE-5414
> Project: HBase
> Issue Type: Sub-task
> Components: client, coprocessors, regionserver
> Reporter: Amitanand Aiyer
> Assignee: Amitanand Aiyer
> Fix For: 0.96.0
>
> Attachments: HBASE-5414.D1749.1.patch
>
>
> HBASE-5203 combines all the different Puts/Deletes into one WALEdit. This is
> required to ensure that we persist the atomic mutation in its enterity and not
> in parts.
> When combined into a single WALEdit, we create one big familyMap that is a combination
> of all the family maps in the mutations. The KV's in this familyMap have no information
> about memstoreTS (it is not yet assigned).
> However, when we apply the mutations to the Memstore (if there are no failures) we end up
> incrementing the memstoreTS for each operation.
> This can lead to the client seeing different order of operations -- depending on weather or
> not there was a RS crash/restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira