You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Alex Feinberg (JIRA)" <ji...@apache.org> on 2012/05/11 20:20:49 UTC
[jira] [Created] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Alex Feinberg created HBASE-5991:
------------------------------------
Summary: Introduce sequential ZNode based read/write locks
Key: HBASE-5991
URL: https://issues.apache.org/jira/browse/HBASE-5991
Project: HBase
Issue Type: Improvement
Reporter: Alex Feinberg
This is a continuation of HBASE-5494:
Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492570#comment-13492570 ]
Matteo Bertozzi commented on HBASE-5991:
----------------------------------------
Hey [~afeinberg], what is the status of this patch?
expecially in relation to HBASE-5494 (prevent multiple table level operation)
In HBASE-5494 all the master methods are synchronous but in trunk createTable(), deleteTable() & co are asynchronous... any thoughts on how/where to handle the unlock?
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510139#comment-13510139 ]
Enis Soztutar commented on HBASE-5991:
--------------------------------------
Backporting this (HBASE-5991) and HBASE-5494 to trunk.
The rational for the read-write lock is that the master-initiate operations (enable/disable, modify, add/remove column) and region splits should not happen in parallel. However, multiple region splits can happen concurrently, so using a read/write lock for the table makes sense. All the table operations initiated from the master take a write lock. And the region splits take a read lock.
In this patch, we will only be doing the master write lock parts. I'll open another issue to track the read locks for region splits.
One discussion that happened offline in todays HBase hackaton was to use the curator library from netflix, which already has (supposedly) correct implementations for many zookeeper recipes (https://github.com/Netflix/curator). The 89-fb patch already contains a read/write zk lock that is implemented from the zk recipe in pseudo-code, so we have to chose one way or the other.
I can go ahead and rebase this patch with the Curator's version to see whether we can go with it. If we want to continue the discussion on whether to go with curator or not, we can alternatively create a separate jira.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408832#comment-13408832 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Hi Ted,
Sorry, I haven't followed up on this -- I have been busy.
Yes, I still intend to do work on this. Unless you've started working on it, I can finish it: I've already started and have a design in mind.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510096#comment-13510096 ]
Enis Soztutar commented on HBASE-5991:
--------------------------------------
Phabricator indicates that it has been committed to 0.89-fb. I'm working on a trunk port for this.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu reassigned HBASE-5991:
-------------------------------------
Assignee: (was: Alex Feinberg)
I seems Alex is not working on this.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492584#comment-13492584 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Hi Matteo,
It's committed to 89-fb. Stack is working to port this to trunk.
Re: asynchronous. I'll have to take a look at trunk first, but couldn't unlock be done using a callback/ListenableFuture (in which case the unlock will be in both .onFailure() and .onSuccess())?
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421070#comment-13421070 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Hi Jesse,
1) Re: progress -- I have another issue I am working on this week (related to log splitting) -- but let me see if I can shuffle things around and get this finished this week (I'll let you know an ETA): I'll let you know (at the latest) sometime tomorrow (hopefully earlier) if I can get this done this week. If not, I'll let you know so you could work on this.
2) Re: locking the table to read only -- Sequential locks let us introduce read-write lock for metadata, so I think it will also be possible to introduce a write lock for data itself. Good suggestion.
Thanks,
- Alex
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510162#comment-13510162 ]
stack commented on HBASE-5991:
------------------------------
Thanks for the jcommon pointer Alex. Studying it now....
Agree on avoiding multiple implementations. We're talking 0.96, the singularity, and thinking if we are going to switch, this is the time to do it. Need a good reason to make the change or as Jon Hsieh says, 3 or 4 good medium size reasons... Talking about it at hackathon.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422909#comment-13422909 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Mostly done in terms of implementing the locks itself based on the recipe (with recoverable zooKeeper). Should have this integrated into HMaster (in place of my DistributedLock code) and have a diff ready soon.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HBASE-5991:
---------------------------------
Fix Version/s: 0.89-fb
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Feinberg reassigned HBASE-5991:
------------------------------------
Assignee: Alex Feinberg
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397811#comment-13397811 ]
Zhihong Ted Yu commented on HBASE-5991:
---------------------------------------
@Alex:
Are you working on this issue ?
Thanks
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424190#comment-13424190 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
All unit tests for exclusion functionality passing. Few issues remaining in handling custom specified timeouts. Will iron out and post a diff early next week.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422458#comment-13422458 ]
Jesse Yates commented on HBASE-5991:
------------------------------------
@Alex Sweet, looking forward to it.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510129#comment-13510129 ]
stack commented on HBASE-5991:
------------------------------
[~afeinberg] Hey boss... we're sitting in same room at the moment (hbase hackathon going on up here in SF).
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423686#comment-13423686 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Got tests for write lock passing (verifying that write locks excludes other writers). Now writing test for read locks (verifying that write locks exclude readers, but that readers do not exclude other reads).
After that the tasks are to integrate misc functionality (printing information on lock owners) into the code, clean up, and then replace DistributedLock with WriteLock and run full end-to-end tests. Will put up a diff once this is done.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428382#comment-13428382 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
(Please disregard last comment, wrong diff linked.)
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427796#comment-13427796 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Integrated and fully working. Will add Javadoc and put up a diff shortly.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421057#comment-13421057 ]
Jesse Yates commented on HBASE-5991:
------------------------------------
@Alex not to press too much, but any progress? Over on HBASE-6055 it would be really nice to not just have locks on master operations, but full-fledged table metadata and/or read/write locks. That way an operation that spans regionservers could lock a table (checkable at the regionserver as well), preventing things like:
* splits
* merges
* region reassignment
Also, it would be nice if the lock would be flexible enough to be used to lock a table down to a read-only state (so any writes get blocked/regjected until the table is unlocked). I think something along the lines of what was up in HBASE-5494 would be the right way to go (leased based lock control and watcher in ZK), perhaps with the addition of some lock listeners.
If you are busy with other stuff, I'd be glad to work on it.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427143#comment-13427143 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Unit test with custom timeout passing. Now working to integrate this and preparing a diff.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421072#comment-13421072 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Just spoke to Liyin about this -- I'll work on this week and will post an update (and hopefully a diff) by Thursday.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510103#comment-13510103 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
[~enis] Nice, thanks for jumping into this! Make sure to sync up with [~saint.ack@gmail.com] -- he also wanted to work on protobuf conversion/trunk patch.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510158#comment-13510158 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Oh great! Didn't know there was a hackathon going on.
I actually looked at curator code, but found it a bit overkill for this specific use case (particularly because we already had an implementation of the recovery logic in RecoveringZooKeeper -- so we'd either have to migrate wholesale or keep two implementations of the same code). I did borrow a few ideas from there (even though I didn't follow the exact logic used, I believe), however, so it wasn't purely from the wiki + scratch.
After I wrote this patch, we've also open sourced a library that Puma and several other apps use to handle ZK. They use a slightly different version of RecoveringZooKeeper, however, that doesn't embed additional information into the data (like we do).
https://github.com/facebook/jcommon/tree/master/zookeeper/src/main/java/com/facebook/zookeeper
There are implementations of different recipes there as well. I have no strong preference on which part is better, there's a lot I like about curator (I would seriously consider using it for something I start from scratch). I'd just avoid having multiple implementations of the same ZK abstraction in the codebase.
One approach could be to just implement the interfaces with curator and then run this through the unit tests.
Good luck! Feel free to put me on the diff(s). I am even more excited about what could now be done on top of these abstractions.
- af
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Feinberg reassigned HBASE-5991:
------------------------------------
Assignee: Alex Feinberg
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Alex Feinberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422437#comment-13422437 ]
Alex Feinberg commented on HBASE-5991:
--------------------------------------
Working on this right now.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based
read/write locks
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510937#comment-13510937 ]
Enis Soztutar commented on HBASE-5991:
--------------------------------------
@Alex thats for the pointers and the feedback. jcommons seems interesting, but one motivator for curator was that it already has reentrant read/write lock, barriers (for hbase snapshots), and queues (for log splitting?). I did not see those recipes in jcommons.
With your patch already having locks, John's barrier implementation, and the rest of the stuff, there is definitely less reason to switch, but agreed with Stack in that this is the perfect time to switch, if we chose to :)
bq. One approach could be to just implement the interfaces with curator and then run this through the unit tests.
Agreed. That was my intention as well.
> Introduce sequential ZNode based read/write locks
> --------------------------------------------------
>
> Key: HBASE-5991
> URL: https://issues.apache.org/jira/browse/HBASE-5991
> Project: HBase
> Issue Type: Improvement
> Reporter: Alex Feinberg
> Assignee: Alex Feinberg
> Fix For: 0.89-fb
>
>
> This is a continuation of HBASE-5494:
> Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira