You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Liyin Tang (JIRA)" <ji...@apache.org> on 2012/10/03 03:16:07 UTC
[jira] [Created] (HBASE-6930) [89-fb] Avoid acquiring the same row
lock repeatedly
Liyin Tang created HBASE-6930:
---------------------------------
Summary: [89-fb] Avoid acquiring the same row lock repeatedly
Key: HBASE-6930
URL: https://issues.apache.org/jira/browse/HBASE-6930
Project: HBase
Issue Type: Bug
Reporter: Liyin Tang
When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same
row lock repeatedly
Posted by "liang xie (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481168#comment-13481168 ]
liang xie commented on HBASE-6930:
----------------------------------
[~liyin], seems we need to open an issue for trunk as well, right? I compared "Revision: 1393468" in fb-89 branch and current trunk code in roughly， it looks trunk code need to have the similar optimization as well.
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same
row lock repeatedly
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481171#comment-13481171 ]
Lars Hofhansl commented on HBASE-6930:
--------------------------------------
The title/description here and the patch (and review) do not seem to mingle...
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6930) [89-fb] Avoid acquiring the same row
lock repeatedly
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6930:
-------------------------------
Attachment: D5841.2.patch
mbautin updated the revision "[jira] [HBASE-6930] [89-fb] Fix TestThriftServerLegacy: notifyAll should be inside synchronized block, and a null checkAndMutate expected value should be handled correctly".
Reviewers: Kannan, Liyin, Karthik, JIRA
Adding ThriftServerRunner fixes
REVISION DETAIL
https://reviews.facebook.net/D5841
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerLegacy.java
To: Kannan, Liyin, Karthik, JIRA, mbautin
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same
row lock repeatedly
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468929#comment-13468929 ]
Phabricator commented on HBASE-6930:
------------------------------------
Liyin has accepted the revision "[jira] [HBASE-6930] [89-fb] Fix TestThriftServerLegacy: notifyAll should be inside synchronized block, and a null checkAndMutate expected value should be handled correctly".
Thanks Mikhail !
REVISION DETAIL
https://reviews.facebook.net/D5841
BRANCH
fix_locked_rows_v2
To: Kannan, Liyin, Karthik, JIRA, mbautin
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6930) [89-fb] Avoid acquiring the same row
lock repeatedly
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6930:
-------------------------------
Attachment: D5841.1.patch
mbautin requested code review of "[jira] [HBASE-6930] [89-fb] Fix TestThriftServerLegacy: notifyAll should be inside synchronized block".
Reviewers: Kannan, Liyin, Karthik, JIRA
There were a couple of reasons why TestThriftServerLegacy has been failing recently in the HBase 89-fb branch:
- rHBASEEIGHTNINEFBBRANCH1393468 was calling notifyAll outside a synchronized block
- rHBASEEIGHTNINEFBBRANCH1391219 changed the meaning of a null expected value passed to checkAndMutate but that was not reflected in the Thrift handler
TEST PLAN
Run TestThriftServerLegacy
REVISION DETAIL
https://reviews.facebook.net/D5841
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/13833/
To: Kannan, Liyin, Karthik, JIRA, mbautin
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same
row lock repeatedly
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469004#comment-13469004 ]
Phabricator commented on HBASE-6930:
------------------------------------
mbautin has closed the revision "[jira] [HBASE-6930] [89-fb] Fix TestThriftServerLegacy: notifyAll should be inside synchronized block, and a null checkAndMutate expected value should be handled correctly".
CHANGED PRIOR TO COMMIT
https://reviews.facebook.net/D5841?vs=19251&id=19293#differential-review-toc
REVISION DETAIL
https://reviews.facebook.net/D5841
COMMIT
https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1393839
To: Kannan, Liyin, Karthik, JIRA, mbautin
> [89-fb] Avoid acquiring the same row lock repeatedly
> ----------------------------------------------------
>
> Key: HBASE-6930
> URL: https://issues.apache.org/jira/browse/HBASE-6930
> Project: HBase
> Issue Type: Bug
> Reporter: Liyin Tang
> Attachments: D5841.1.patch, D5841.2.patch
>
>
> When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again.
> So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira