You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Kadir OZDEMIR (JIRA)" <ji...@apache.org> on 2019/04/14 21:10:00 UTC
[jira] [Comment Edited] (PHOENIX-5156) Consistent Mutable Global
Indexes for Non-Transactional Tables
[ https://issues.apache.org/jira/browse/PHOENIX-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16817420#comment-16817420 ]
Kadir OZDEMIR edited comment on PHOENIX-5156 at 4/14/19 9:09 PM:
-----------------------------------------------------------------
You got it right. If preparing index mutations for a batch of mutations takes on average less than 0.5ms (at most 1ms), then the thread for this batch sleeps for 1ms.
Assume that two batches of mutations concurrently want to update the same row X and plus some other rows on the data table. One of them, batch 1, acquires the row locks for its rows and the other, batch 2, is waiting for the row lock for X.
Batch 1 goes ahead and prepares the index updates for its rows blazingly fast and finishes this in less than 1ms (I think this will be a very rare event) . It releases the row locks to do the RPC calls for index updates (i.e., the first write phase).
Batch 2 now acquires its locks in the same ms and uses the same timestamp as batch 1 did.
As you can see, the order in which the updates happens on row X on the data table can be different than that on an index table in this case. Since these updates use the same timestamp, HBase cannot order them correctly.
The solution I suggested for this problem is that Mutation 1 detects that the current ms is the same as the ms just before the index update preparation, and thus, sleeps for 1ms before releasing the row locks so that Mutation 2 gets a different timestamp.
was (Author: kozdemir):
You got it right. Not only two batches of mutations need to arrive on the same millisecond for at least one common row but also the one acquiring the row locks first needs to take on average less than 0.5ms (at most 1ms) to prepare the index mutations.
Assume that two batches of mutations concurrently want to update the same row X and plus some other rows on the data table. One of them, batch 1, acquires the row locks for its rows and the other, batch 2, is waiting for the row lock for X.
Batch 1 goes ahead and prepares the index updates for its rows blazingly fast and finishes this in less than 1ms (I think this will be a very rare event) . It releases the row locks to do the RPC calls for index updates (i.e., the first write phase).
Batch 2 now acquires its locks in the same ms and uses the same timestamp as batch 1 did.
As you can see, the order in which the updates happens on row X on the data table can be different than that on an index table in this case. Since these updates use the same timestamp, HBase cannot order them correctly.
The solution I suggested for this problem is that Mutation 1 detects that the current ms is the same as the ms just before the index update preparation, and thus, sleeps for 1ms before releasing the row locks so that Mutation 2 gets a different timestamp.
> Consistent Mutable Global Indexes for Non-Transactional Tables
> --------------------------------------------------------------
>
> Key: PHOENIX-5156
> URL: https://issues.apache.org/jira/browse/PHOENIX-5156
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.13.0, 4.14.0, 5.0.0, 4.14.1
> Reporter: Kadir OZDEMIR
> Assignee: Kadir OZDEMIR
> Priority: Major
> Attachments: PHOENIX-5156.master.001.patch, PHOENIX-5156.master.002.patch, PHOENIX-5156.master.003.patch, PHOENIX-5156.master.004.patch, PHOENIX-5156.master.005.patch, PHOENIX-5156.master.006.patch, PHOENIX-5156.master.007.patch, PHOENIX-5156.master.008.patch
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Without transactional tables, the mutable global indexes can get easily out of sync with their data tables in Phoenix. Transactional tables require a separate transaction manager, have some restrictions and performance penalties. This issue is to have consistent mutable global indexes without the need for using transactional tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)