You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@bookkeeper.apache.org by merlimat <gi...@git.apache.org> on 2017/05/11 18:31:12 UTC

[GitHub] bookkeeper pull request #153: BOOKKEEPER-1066: Introduce GrowableArrayBlocki...

GitHub user merlimat opened a pull request:

    https://github.com/apache/bookkeeper/pull/153

    BOOKKEEPER-1066: Introduce GrowableArrayBlockingQueue

    In multiple places, (eg: journal, ordered executor, etc..), we are using `LinkedBlockingQueue` instances to pass objects between threads.
    
    The `LinkedBlockingQueue` differs from the `ArrayBlockingQueue` in that it doesn't require to define a max queue size, though, being implemented with a linked list, it requires to allocates list nodes each time an item is added.
    
    We can use a `GrowableArrayBlockingQueue` that behaves in the same way as the `LinkedBlockingQueue`, but it's implemented with an array that can be resized when the queue reaches the capacity.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/merlimat/bookkeeper growable-blocking-queue

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/bookkeeper/pull/153.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #153
    
----
commit e98cf0b9fa77b7b79fec791b295b45abb0a2b5d4
Author: Matteo Merli <mm...@apache.org>
Date:   2017-05-11T18:29:20Z

    BOOKKEEPER-1066: Introduce GrowableArrayBlockingQueue

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] bookkeeper issue #153: BOOKKEEPER-1066: Introduce GrowableArrayBlockingQueue

Posted by merlimat <gi...@git.apache.org>.

Github user merlimat commented on the issue:

    https://github.com/apache/bookkeeper/pull/153
  
    @jvrao I don't have a benchmark for this specific change, though I agree it might make sense to add one.
    
    The points here are:
     * In some places we are using `LinkedBlockingQueue` which uses the same head/tail mutex schema but with a linked list instead of the array
     * I don't expect the throughput / cpu utilization to be different between the 2
     * This implementation doesn't require to create a LinkedList node for each value, so it creates no garbage. This can be significant since we are enqueuing quite a bit of items (Journal thread, force-sync thread, OrderedSafeExcutor). For each BK request there were several of these (cannot remember exact number, though I believe ~3 in Bookie and another ~3 in client)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] bookkeeper pull request #153: BOOKKEEPER-1066: Introduce GrowableArrayBlocki...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/bookkeeper/pull/153


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] bookkeeper issue #153: BOOKKEEPER-1066: Introduce GrowableArrayBlockingQueue

Posted by jvrao <gi...@git.apache.org>.

Github user jvrao commented on the issue:

    https://github.com/apache/bookkeeper/pull/153
  
    Interesting and on the paper sounds great. But do we have any perf testing or benchmark? I believe we need to have a benchmark / ref to run and compare any perf improvements. I have seen extremely interesting results for changes which made absolute sense on paper. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---