[jira] [Commented] (BOOKKEEPER-25) Implementation of the JMS specification and API

["Matthieu Morel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066861#comment-13066861 ] 

Matthieu Morel commented on BOOKKEEPER-25:
------------------------------------------

Here is an update on a several points we've been discussing offline.

They correspond to clarifications and required modifications to hedwig.

* Authentication 
Authentication can be used for creating Connection objects. A Connection object represents the client's active connection to the JMS broker.
The JMS specification suggests an optional basic username / password authentication scheme. 
More robust authentication schemes may be implemented but this is outside of the scope of this current task.
However, we should allow for authentication schemes to be pluggable into the existing framework.


* Point to point (PTP)
A message is only received once by 1 consumer only. The JMS spec does not define how messages are distributed when there is more than 1 consumer on the queue.

This communication model is currently not provided by Hedwig.

The mechanism used for pub-sub can be reused, with the following modification: instead of sending queued messages to every subscriber, a message should only be sent to 1 subscriber. Only after message acknowledgement from this subscriber, should the next message of the queue be sent. The selection of the receiving subscriber should be pluggable (round-robin, or based on some metrics from subscribers)


* Messages
JMS messages a structured as follow:
- header
- properties
- content
Messages can be filtered by message selectors. Message selectors apply SQL queries on header and properties of JMS messages.
We must therefore extend current Hedwig messages to provide direct access to these additional metadata elements. We must also write an interpreter of SQL queries. 
We should try to provide a generic mechanism that can be applied directly on Hedwig messages, and that is not specific to the JMS wrapper.

In addition, there are several types of message bodies : text, object, stream, map, bytes . We must provide conversion facilities serializing and deserializing these bodies into expected types, and provide getters and setters according to the type of message.


* JNDI configuration
JNDI is used to retrieve connection factories, as well as statically defined queues and topics (Destination objects).
We must provide a mechanism to fetch those properties from a JNDI directory, and instantiate corresponding objects on the client side.
Some JMS implementations use a simple text file for storing JNDI properties.


* Development tasks
I updated the attached mind map, in order to identify a sequence of relatively independent tasks that we'll have to complete. 
The idea is to :
1. implement minimum interfaces in order to provide minimal pub/sub communication through the JMS API. That would be a milestone
2. add the remaining functionalities (messages, filtering, subscription modes, auth etc...) 
3. add point to point communication
4. validation + perf tests




> Implementation of the JMS specification and API
> -----------------------------------------------
>
>                 Key: BOOKKEEPER-25
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-25
>             Project: Bookkeeper
>          Issue Type: Task
>            Reporter: Matthieu Morel
>         Attachments: JMS implementation.pdf, JMS implementation.pdf
>
>
> Here is a proposal for implementing JMS on top of Hedwig.
> Overview:
> ---------
> After comparing the JMS spec and Hedwig's current implementation, it appears that Hedwig provides most concepts of the JMS spec. The work will therefore mostly be implementing wrappers around Hedwig's primitives.
> There is no 1 to 1 mapping most of the time though, and there are some subtleties in the JMS spec that we must be cautious about. There are also things that don't exist in Hedwig and that we'll have to add. 
> For instance, "administered objects", the different kinds of standard messages, or a parser for the SQL-like selector queries. These are marked with a triangular flag on the attached mindmap.
> The JMS spec also specifies support for distributed transactions (XA) but this part is optional. I would suggest to leave that for later, if there is any request for it.
> Design:
> -------
> I suggest to separate all JMS-related code in a separate project (as hedwig-client and hedwig-protocol). 
> We'll therefore make sure that Hedwig-server is JMS agnostic, and it will later be possible to use the same approach for implementing AMQP or JMS2, without mixing all concepts.
> Development:
> -----------
> An iterative approach is well suited. We could start with fundamentals, and progressively add some meat: 
> 1. messages, connections and sessions
> 2. basic messaging with selectors
> 3. all cases of messaging models
> 4. validation tests
> 5. performance tests (I checked and unfortunately, there does not seem to be any "free" standard for benchmarking jms implementation...)
> Each iteration would involve adding regression tests and doing a review.
> About the time this would take:
> My rough estimation (not being very familiar with Hedwig's codebase): in the order 1 or 2 weeks for each of these iterations.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira