You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Timothy Steffens <ti...@yahoo.com.INVALID> on 2017/11/07 22:10:25 UTC

JMSStreamer enhancement -- needs to support multiple IgniteDataStreamers.

Dev -
I have a use case that is not able to be addressed with the existing JMSStreamer functionality so I would like to propose an enhancement.
The use case is such that I have a series of database tables and am writing messages into a shared JMS queue for each table update. These messages are then picked up by the JMSStreamer, converted into various object types (based on an indicator within the message), and written into a shared Ignite cache that holds all object types. I would like to enhance the JMSStreamer so that it accepts multiple IgniteDataStreamer instances so that it can stream each object type into its own corresponding cache rather than having to stream into a shared, generic cache.
My approach will be to take advantage of the inherited StreamAdapter functionality within the JMSStreamer. Note that my changes will be backwards compatible so that other streamers are not broken. Additionally, these changes will streamline the message processing functionality which is duplicated between the StreamAdapter and JMSStreamer today.
StreamAdapter   
   - Update the StreamAdapter to accept a list of IgniteDataStreamers - the getStreamer() method will be kept for backwards compatibility and I will add getStreamers().
   
   - The StreamAdapter's addMessage() method now has to interrogate the Message type to determine onto which streamer to place the message. This can be done by providing a custom map that relates the message type to a specific streamer.
JMSStreamer   
   - Update the JMSStreamer add a setStreamers() method that takes a list of streamers - the setStreamer() method will be left for backwards compatibility.
   
   - The start() method is updated to verify the list of streamers is not null and to verify that a Single/MultipleTupleExtractor was provided instead of a MessageTransformer.
   
   - Mark the setTransformer() method as deprecated.
   
   - The executor.execute() method is updated to use the inherited addMessage() method from StreamAdapter.
   
   - Delete the JMSStreamer private processMethod() after refactoring to use the StreamAdapter addMessage() method.
   
   - Remove the MessageTransformer property from the JMSStreamer because it will now use the Single/MultipleTupleExtractor within the StreamAdapter - these two classes do basically the same thing. The MessageTransformer class can then be deprecated and go away as it is not used anywhere else in the code base.
MessageTransformer   
   - Mark as deprecated.

Please review and let me know how to proceed.

Thank you.