You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by jyates <gi...@git.apache.org> on 2014/03/18 19:56:26 UTC

[GitHub] incubator-phoenix pull request: Removing holding log locks for upd...

GitHub user jyates opened a pull request:

    https://github.com/apache/incubator-phoenix/pull/21

    Removing holding log locks for updates

    Now we do all the writes during postBatchMutate. This effectively ends up
    serializing writes under the mvcc number (since updates wait on their
    write to be visible before returning), but ensures that the correct locks
    are released at the correct time.
    
    Because of the above locking/mvcc situation, indexing needs to be part of
    hbase core; it would allow it to be a bit more fine grained in the use of
    locks mvcc. However, until that point, this is the correct solution - regardless
    of whether or not its faster/slower.
    
    The change is necessary because the LogRoll locks caused a deadlock on the
    log roll/cache flush on server shutdown in our (Salesforce.com) internal
    environment. Essentially, we think an exception was thrown while writing
    to the WAL during batchMutate. This then caused the INDEX_UPDATE_LOCK
    to never be released. Since this lock is waited on during logArchive,
    the archiver was blocked there
    
    Also, this update removes special handling of deletes. Since HBase 0.96,
    Deletes and Puts both go through HRegion#batchMutate, so we can get rid
    of the special casing for preDelete. This doesn't apply to any branch
    running on HBase 0.94

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jyates/incubator-phoenix indexing-no-locks

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-phoenix/pull/21.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21
    
----
commit 36631458543222b5b93875d5341d1a695aa9bb98
Author: Jesse Yates <jy...@apache.org>
Date:   2014-03-18T02:58:00Z

    Removing holding log locks for updates
    
    Now we do all the writes during postBatchMutate. This effectively ends up
    serializing writes under the mvcc number (since updates wait on their
    write to be visible before returning), but ensures that the correct locks
    are released at the correct time.
    
    Because of the above locking/mvcc situation, indexing needs to be part of
    hbase core; it would allow it to be a bit more fine grained in the use of
    locks mvcc. However, until that point, this is the correct solution - regardless
    of whether or not its faster/slower.
    
    The change is necessary because the LogRoll locks caused a deadlock on the
    log roll/cache flush on server shutdown in our (Salesforce.com) internal
    environment. Essentially, we think an exception was thrown while writing
    to the WAL during batchMutate. This then caused the INDEX_UPDATE_LOCK
    to never be released. Since this lock is waited on during logArchive,
    the archiver was blocked there
    
    Also, this update removes special handling of deletes. Since HBase 0.96,
    Deletes and Puts both go through HRegion#batchMutate, so we can get rid
    of the special casing for preDelete. This doesn't apply to any branch
    running on HBase 0.94

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: Removing holding log locks for upd...

Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on the pull request:

    https://github.com/apache/incubator-phoenix/pull/21#issuecomment-37978013
  
    Looks good, Jesse. Thanks for tracking this down (doesn't sound like it was easy). Please check this in and check into 3.0 with the modification you mentioned. We'll do a perf run once it's in to see the impact it has.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---