You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucy.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/02/20 16:38:44 UTC
[lucy-issues] [jira] [Commented] (LUCY-323) Improve locking code
[ https://issues.apache.org/jira/browse/LUCY-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874781#comment-15874781 ]
ASF GitHub Bot commented on LUCY-323:
-------------------------------------
GitHub user nwellnhof opened a pull request:
https://github.com/apache/lucy/pull/53
Improve locking code
Fixes [LUCY-323](https://issues.apache.org/jira/browse/LUCY-323).
- Major rewrite of FilePurger, making it do less work.
- Eliminate global deletion lock, making PolyReader and FilePurger never wait for a lock.
- Redesign Lock API in preparation for native locks.
- Remove LockFactory.
- Fix locking bug with non-default merge lock timeout.
- Fix locking bug with concurrent BackgroundMergers.
- Port some tests to C.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nwellnhof/lucy improve-locking
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucy/pull/53.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #53
----
commit 2051451de456f51e35345f12206ca4cb0040dd7c
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-15T04:10:07Z
Separate purging of snapshots and aborted merges
Purge aborted merge when starting an Indexer/BGMerger session and
purge snapshots at the end of Commit.
Rework FilePurger to use a single hash for purged and spared files
when purging snapshots. Optimize Folder handling assuming that
Snapshots only contain entries local to the index folder.
Make sure to list the contents of the real directory, not the
virtual compound file directory. CFReaderDirHandle lists both real
and virtual files which might be a bug in itself.
Port BGMerger tests to C.
TODO: Move Perl BGMerge test to t/binding.
commit bb83d0d77bf99ed09b2bbad782c3c1d3b94f04a9
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-16T14:58:45Z
Merge SharedLock into LockFileLock
Supplying the lock type when requesting a lock better matches other
file lock APIs. Merging the implementations allows to convert
Maybe_Delete_File into a static function.
Make shared and exclusive locks check for locks of the other type
except for exclusive locks created with `exclusive_only`.
Clear_Stale is broken for shared locks but will be removed with the
following commit.
commit 1a8e02df0a0257e4ba70de4a41504507b0ba8a47
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-16T15:47:03Z
Always check for stale locks in Is_Locked
Note that checking for stale locks isn't necessary when requesting
shared locks.
commit 3e2298650d8fc0213ce25078cadeb4f22d45bb5f
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-16T16:47:33Z
Remove LockFactory and make Lock a private class
If there are users who really use custom locks, this can be reverted.
commit 79161a4b233ce3570ecf4c48745b469ccb50aec7
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T12:53:59Z
Don't allow double obtain/release
commit 5270b988c477a2e1c110e2069c66bb2d3f27a60f
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T16:19:43Z
Rework snapshot and deletion locks
Request exclusive or shared snapshot locks instead of the global
deletion lock. This removes the need to retry lock requests and
reduces contention.
commit 4e6e5bbf2814e8e2a824b5ae1b91d068233cd248
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T17:25:35Z
Improve merge lock handling
Don't check the merge lock with Is_Locked in
FilePurger_Purge_Aborted_Merge. This prevented background mergers from
cleaning up because they already held the merge lock. Require that
callers hold the merge lock instead.
Don't check the merge lock with Is_Locked in Indexer_init. Since we
hold the write lock, and the merge.json file from aborted merge has
been removed, the presence of a merge.json file is a reliable indicator
that it belongs to an active background merger. Move the merge lock
request form S_maybe_merge to Indexer_init.
Also, S_maybe_merge used to Obtain the merge lock instead of merely
Requesting it. Waiting for a potentially long-running background merger
slows things down unnecessarily and increases the chance for write lock
contention. This didn't affect the default configuration which has a
zero merge lock timeout.
commit de14d4ca31ab27fc1569966c1b182fa4829afd40
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T17:34:31Z
Change lock acquitision order in BGMerger_init
Obtain the merge lock before the write lock so that an already running
background merger won't abort when failing to acquire the write lock
during commit.
Release the merge lock at the very end of BGMerger_Commit. It doesn't
really matter when we release the merge lock after we acquired the
write lock in BGMerger_Prepare_Commit. I think it makes the code
clearer to release it at the very end of BGMerger_Commit.
commit 38a33badb310ba9fea5bf9a923277da4a82ec369
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T18:13:59Z
Remove Lock_Is_Locked
commit e06cdbf63d3036c3a33e5265444d827065539e92
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T19:26:13Z
Don't hide I/O errors behind LockErr
Only return a LockErr if there's lock contention, not on unrelated
I/O errors.
commit 35388cd22600cb3c202cf11c8c9afd661c8f4fe4
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-17T20:38:56Z
Port LockFileLock tests to C
commit d23b560dc34d1c1cbc731fdc745b4e72324d8717
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-19T13:09:02Z
Release locks on destruction
commit 2bd2bc611325ee643f325ecfd11d5227319949a8
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-19T15:21:00Z
Move host ivar from Lock to LockFileLock
commit 6e8538a8834a49da7e8cce8b94decc130467b0f3
Author: Nick Wellnhofer <we...@aevum.de>
Date: 2017-02-19T16:01:50Z
Move LockFileLock to separate file
----
> Improve locking code
> --------------------
>
> Key: LUCY-323
> URL: https://issues.apache.org/jira/browse/LUCY-323
> Project: Lucy
> Issue Type: Improvement
> Reporter: Nick Wellnhofer
>
> The locking code can be improved in a couple of places. See https://lists.apache.org/thread.html/c1a98bede3b2227a83d7814b575dfd2b2e64b51670e0ecb7a7290e90@%3Cdev.lucy.apache.org%3E
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)