Camel-Quickfixj 2.9.3 - Ordering issue

[vcheruvu <v_...@hotmail.com>]

Hello Camel Riders,

  I have used Camel-Quickfixj component in our application to consume and
publish fix messages. Our application consumes FIX 4.2 messages from VM
queue with 10 concurrent consumers.   Each consumer takes the FIX message of
the VM queue, transforms the message to our internal FIX message structure
and publishes on to another FIX session. Please see routebuilder definition
below.  I can tell in our logs that FIX messages are consumed in order. I
also verified that transformation bean class have processed FIX messages in
order and sent it  to the pipeline for QuickFixJ publishing in order as
well. However, after the transformation bean class,  publishing of new
transformed FIX messages are out of order (I could tell by looking at
Quickfix/J message store that fix messages are sent in different order.
Please note this is not FIX sequence number issue.).  

 I understand obvious solution is make multiple consumer of VM to single
consumer but this could seriously effects the speed of publishing fix
messages. To be honest, I am little unclear how this route works when it is
parallel processing, i.e if 10 concurrent consumers of VM, does that mean 10
threads transforming messages and invoking quickfixj's session.send() at
same time.  Could you please advise how I can ensure ordering is preserved
between transformation and publishing of Quickfix/J when it is parallel
processing? 


Routbuilder setup example in our application

from(vm:FixQueue?concurrentConsumers=10)
.to(bean:transformerBean?method="transformFixMessage")
.to(quickfix:config/session.cfg?sessionID=FIX.4.2:Test->Destination)

Kind regards,
-Vid-



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quickfixj-2-9-3-Ordering-issue-tp5726595.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel-Quickfixj 2.9.3 - Ordering issue

[vcheruvu <v_...@hotmail.com>]

Thanks, Christian. I have implemented Resequencer using the Quickfix fix
object's  sequence number and also added custom logic to preserve sequence
number in exchange object when receiving fix messages from multilple FIX
sessions. It works like a charm... cheers



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quickfixj-2-9-1-Ordering-issue-tp5726595p5726855.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel-Quickfixj 2.9.3 - Ordering issue

[Christian Müller <ch...@gmail.com>]

With parallel processing you cannot ensure ordering out of the box. You
need an additional resequencer to achieve it.

Sent from a mobile device
Am 31.01.2013 05:22 schrieb "vcheruvu" <v_...@hotmail.com>:

> Hello Camel Riders,
>
>   I have used Camel-Quickfixj component in our application to consume and
> publish fix messages. Our application consumes FIX 4.2 messages from VM
> queue with 10 concurrent consumers.   Each consumer takes the FIX message
> of
> the VM queue, transforms the message to our internal FIX message structure
> and publishes on to another FIX session. Please see routebuilder definition
> below.  I can tell in our logs that FIX messages are consumed in order. I
> also verified that transformation bean class have processed FIX messages in
> order and sent it  to the pipeline for QuickFixJ publishing in order as
> well. However, after the transformation bean class,  publishing of new
> transformed FIX messages are out of order (I could tell by looking at
> Quickfix/J message store that fix messages are sent in different order.
> Please note this is not FIX sequence number issue.).
>
>  I understand obvious solution is make multiple consumer of VM to single
> consumer but this could seriously effects the speed of publishing fix
> messages. To be honest, I am little unclear how this route works when it is
> parallel processing, i.e if 10 concurrent consumers of VM, does that mean
> 10
> threads transforming messages and invoking quickfixj's session.send() at
> same time.  Could you please advise how I can ensure ordering is preserved
> between transformation and publishing of Quickfix/J when it is parallel
> processing?
>
>
> Routbuilder setup example in our application
>
> from(vm:FixQueue?concurrentConsumers=10)
> .to(bean:transformerBean?method="transformFixMessage")
> .to(quickfix:config/session.cfg?sessionID=FIX.4.2:Test->Destination)
>
> Kind regards,
> -Vid-
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Quickfixj-2-9-3-Ordering-issue-tp5726595.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>