[jira] Resolved: (QPID-950) Refactor Java Broker

["Rob Godfrey (JIRA)" <qp...@incubator.apache.org>]

     [ https://issues.apache.org/jira/browse/QPID-950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rob Godfrey resolved QPID-950.
------------------------------

    Resolution: Fixed

> Refactor Java Broker
> --------------------
>
>                 Key: QPID-950
>                 URL: https://issues.apache.org/jira/browse/QPID-950
>             Project: Qpid
>          Issue Type: Improvement
>          Components: Java Broker
>    Affects Versions: M2.1
>            Reporter: Marnie McCormack
>            Assignee: Rob Godfrey
>             Fix For: M4
>
>
> Requirements
> ===========
> At the Qpid Java Offsite in December 2007 the Qpid Java Developers identified that the internal queuing model of the Java Broker was an area ripe for some re-implementation.
> In particular the Java Broker uses a "DeliveryManager" on every queue to manage the transfer of messages from the queue to the queues subscribers.  The queue itself was modelled as a queue with no lookahead and the entries in the queue were the messages themselves (messages which may be shared between many queues).
> Particular issues with this model were faced when implementing JMS selectors which require lookahead and out of order dequeuing; and also when items sent to a subscriber are requeued due to the consumer rejecting/rolling back or simply closing before accepting a message.
> High level Design
> ==============
> The developers collectively agreed that a new approach should be taken.  Namely that each queue should be made up of "QueueEntry"s which model the existence of a message on that particular queue.  Further these QueueEntrys should have a state representing their state on that queue (e.g. available, acquired [assigned and delivered to a particular subscriber], deleted etc).  Entries should not be removed from the queue until they had been dequeued with no possibility of return.  Subscribers should use pointers into this queue to keep a knowledge of where they had seen up to.  When deliverying to a subscriber, messages from the queue which were not of interest to a particular Subscriber (e.g. due to Selectors) or which had already been acquired by a different subscription should be skipped.  If a message is returned to the queue then any subscriber which was logically "past" the point of the message that has been requeued should have its pointer reset so that it could examine the requeued message.
> Implementation
> ============
> From the high level design agreed at the F2F the implementation strategy was as follows:
> First the existing code was modified to use a basic QueueEntry implementation rather than AMQMessage for entries in the queues.  This solved the issue of queue specific state having to be held on the AMQMessage.
> Then functionality from the DeliveryManager was moved either to the subscription or to the queue as appropriate.  Different sub-classes of subscription were used to represent different models of consumption (e.g. browsing, different acknowledgement modes.  Flow controlling logic was abstracted and placed into a separate class to allow for migration to 0-10 where flow control is per subscription rather than per channel..
> The major work then was to create a linked-list style queue implementation into which the subscriptions could hold "pointers".  Since the JDK provided collection classes to not expose the internals (i.e. the list elements) through the API, a new class had to be implemented.  From this we could draw heavily from the public domain work that Doug Lea and other have done on the concurrent backport work.  Based on their ConcurrentLinkedQueue an implementation can be made which allows safe concurrent addition to the queue.  This was extended to allow elements in the middle of the queue to be deleted (rather than only being able to consume from the head).
> The broker has two modes for delivery to subscribers "synchronous" and "asynchronous".  When a message is first enqueued the queue (or previously the delivery manager) checks to see if there is any subscriber available who can immediately take the message - and if so the message is immediately sent to that subscriber.  If no such subscriber is found then the message is "queued" and when a subscriber subsequently becomes available an asynchronous delivery thread attempts to deliver all the queued messages to the subscribers that are now active.
> After refactoring the logic changes a little.  Now every message is added to the queue giving a strict ordering between messages.  Since messages must be delivered in order, when a new message arrives, it can only be synchronously delivered to a subscriber if the subscriber in question has already seen all messages prior to the one for which delivery is being attempted (or all messages between the last one the subscriber has seen and the current message have already been acquired or are otherwise of no interest to the subscriber).  If there is no subscriber to which the message can currently be immediately delivered then an synchronous delivery process is started.  Unlike in the previous model it is completely safe for asynchronous and synchronous delivery methods to be in progress concurrently.  the invariants are the same - a subscriber must process the entries in strict order (which is always maintained).  the entry's state must be atomically changed (to acquired) before a subscriber can delivery it.
> The refactoring also introduces a more efficient asynchronous process for processing one subscription at a time.  If a new subscription is created (or an existing one moves to an active from a suspended state) then a process kicks off to move that subscription
> Testing
> ======
> The aim of the refactoring was to leave existing functionality exactly the same while changing the implementation strategy.  Therefore getting the refactored broker to pass the existing tests was the major aim.  A small number of unit tests relating to the behaviour of DeliveryManagers were no longer applicable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.