[jira] [Commented] (BOOKKEEPER-461) Delivery throughput degrades when there are lots of publishers w/ high traffic.

["Sijie Guo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508527#comment-13508527 ] 

Sijie Guo commented on BOOKKEEPER-461:
--------------------------------------

[~ikelly] For 4.2.0, how about keeping the cache implementation here to just use multithreads?

For 4.3.0, we could have a separated jira working on providing a better readahead cache implementation, like using a mature cache library, which we also need to consider memory issue in JVM, one solution is to use on-heap cache implementation like guava cache, the other solution is to use off-heap cache implementation like apache directmemory (http://directmemory.apache.org/).
                
> Delivery throughput degrades when there are lots of publishers w/ high traffic.
> -------------------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-461
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-461
>             Project: Bookkeeper
>          Issue Type: Bug
>            Reporter: Sijie Guo
>            Assignee: Sijie Guo
>             Fix For: 4.2.0
>
>         Attachments: BOOKKEEPER-461.diff, BOOKKEEPER-461.diff, BOOKKEEPER-461.diff, BOOKKEEPER-461.patch, pub_sub_multithreads.png, pub_sub_singlethread.png
>
>
> When running benchmarking over the hub server, found that delivery throughput degrades when there are lots of publishers publishing messages. And the delivery throughput will goes up when there is no publishes.
> This issue is introduced due to ReadAheadCache only runs a single thread. So when the netty workers are busy handling publish requests, they are pushing lots of messages into ReadAheadCache's queue to put them in to read ahead cache. So the readahead cache is busy on updating keys.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira