[jira] [Updated] (ZOOKEEPER-1505) Multi-thread CommitProcessor

["Jay Shrauner (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jay Shrauner updated ZOOKEEPER-1505:
------------------------------------

    Attachment: ZOOKEEPER-1505.patch

- Addressed reviewboard comments.
- Added unit test.
- Bugfix for issue Thawan found with watch resets on read requests in one session racing a write request affecting that watch in another session. Solution taken here is to prevent any read requests at all from running concurrently with a write request. There is room for further improvement, by parsing the request earlier in the pipeline and identifying read requests with watch resets.
                
> Multi-thread CommitProcessor
> ----------------------------
>
>                 Key: ZOOKEEPER-1505
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1505
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: server
>    Affects Versions: 3.4.3, 3.4.4, 3.5.0
>            Reporter: Jay Shrauner
>            Assignee: Jay Shrauner
>              Labels: performance, scaling
>             Fix For: 3.5.0
>
>         Attachments: ZOOKEEPER-1505.patch, ZOOKEEPER-1505.patch
>
>
> CommitProcessor has a single thread that both pulls requests off its queues and runs all downstream processors. This is noticeably inefficient for read-intensive workloads, which could be run concurrently. The trick is handling write transactions. I propose multi-threading this code according to the following two constraints
>   - each session must see its requests responded to in order
>   - all committed transactions must be handled in zxid order, across all sessions
> I believe these cover the only constraints we need to honor. In particular, I believe we can relax the following:
>   - it does not matter if the read request in one session happens before or after the write request in another session
> With these constraints, I propose the following threads
>   - 1    primary queue servicing/work dispatching thread
>   - 0-N  assignable worker threads, where a given session is always assigned to the same worker thread
> By assigning sessions always to the same worker thread (using a simple sessionId mod number of worker threads), we guarantee the first constraint-- requests we push onto the thread queue are processed in order. The way we guarantee the second constraint is we only allow a single commit transaction to be in flight at a time--the queue servicing thread blocks while a commit transaction is in flight, and when the transaction completes it clears the flag.
> On a 32 core machine running Linux 2.6.38, achieved best performance with 32 worker threads for a 56% +/- 5% improvement in throughput (this improvement was measured on top of that for ZOOKEEPER-1504, not in isolation).
> New classes introduced in this patch are:
>     WorkerService (also in ZOOKEEPER-1504): ExecutorService wrapper that makes worker threads daemon threads and names then in an easily debuggable manner. Supports assignable threads (as used here) and non-assignable threads (as used by NIOServerCnxnFactory).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira