You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by stevebate <sv...@stevebate.net> on 2013/02/25 13:09:18 UTC
Re: camel-quickfix RequestReplyExample java.io.IOException
I'll need to refresh my memory since the original code was written four years
ago.
However, I'm sure the example worked at that time or I wouldn't have
submitted or
taken the time to document the example.
When I have some time, I'll set up a Camel development environment to take a
look at
the issue. In the meantime, I have a few questions.
You say that the QuickFIX/J component doesn't support the InOut MEP but
there's a unit
test to verify that it does (QuickfixjConsumerTest.processInOutExchange).
I'm assuming
that unit test is passing. Is that true?
Looking at the SVN trunk, it appears that the RequestReplyExample is still
incorrectly
correlating the request and not the reply. It should be correlating with the
ExecutionReport
and not the OrderStatusRequest. Is that change reversed on a branch
somewhere?
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728097.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi Steve,
Sorry for the late answer but just for the record and also maybe other users
who may follow this thread:
The reason for the behavior you were observing has nothing to do with the
MEP itself but a bug being already fixed:
https://issues.apache.org/jira/browse/CAMEL-5979
So that if yo would try the same now (e.g on the trunk code) then the
behaviour you're describing below is not given anymore.
Babak
stevebate wrote
> On Feb 27, 2013, at 3:12 AM, "Babak Vahdat [via Camel]" <
> ml-node+s465427n5728204h85@.nabble
> > wrote:
>
>> Hi Steve
>>
>> Thanks for looking into this. Unfortunately the provided patch causes
>> regressions by some pre-existing tests of this component. I've provided
>> the details of this into the ticket itself, so let's continue the
>> conversation inside JIRA itself to avoid making noise here @ the user
>> forum.
>
> Sorry. I should have run the unit tests before submitting the patch. I've
> submitted an updated patch with changes to the unit test mock setup for
> the new Exchange copy behavior and the sessionID reversal in the
> MessagePredicate. I agree we can continue any patch-related discussion
> using Jira. However, the following discussion might be of general
> interest.
>
>> And regarding your surprise maybe the following thread could be helpful
>> for a better understanding:
>>
>> http://camel.465427.n5.nabble.com/About-what-to-do-with-the-Response-retrieved-through-a-Producer-when-the-Exchange-is-NOT-out-capable-td5713946.html
>>
>> In general if the MEP is InOnly and a service invocation (through a Camel
>> Producer) returns a reply, then a good practice is to set the reply as
>> the body of the IN message. This would make the next processor in chain
>> to make use of this IN object as it say here:
>
> This is confusing to me because the Exchange is now InOut, not InOnly. I
> did some experimentation and found that this route,
>
>
> from("quickfix:examples/inprocess.cfg?sessionID=FIX.4.2:MARKET->TRADER&exchangePattern=InOut")
>
> .filter(header(QuickfixjEndpoint.MESSAGE_TYPE_KEY).isEqualTo(MsgType.ORDER_STATUS_REQUEST))
> .bean(new MarketOrderStatusService())
> .bean(new QuickfixjMessageJsonPrinter());
>
> required the Exchange copy for the InOut MEP because the inbound message
> is overwritten by subsequent processors. However,
> the following route does not require the copy (the
> QuickfixjMessageJsonPrinter bean has been removed).
>
>
> from("quickfix:examples/inprocess.cfg?sessionID=FIX.4.2:MARKET->TRADER&exchangePattern=InOut")
>
> .filter(header(QuickfixjEndpoint.MESSAGE_TYPE_KEY).isEqualTo(MsgType.ORDER_STATUS_REQUEST))
> .bean(new MarketOrderStatusService());
>
> Does this mean that the QuickfixjConsumer is sometimes required to copy
> the inbound Exchange (InOut MEP) and sometimes not,
> depending on the route components that follow it? If so, the safe behavior
> is to always copy the Exchange, but it seems that it should
> logically not need to do that for an InOut MEP.
>
>> if there is no out message then the in message is used instead
>> http://camel.apache.org/using-getin-or-getout-methods-on-exchange.html
>>
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5731828.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by stevebate <sv...@stevebate.net>.
On Feb 27, 2013, at 3:12 AM, "Babak Vahdat [via Camel]" <ml...@n5.nabble.com> wrote:
> Hi Steve
>
> Thanks for looking into this. Unfortunately the provided patch causes regressions by some pre-existing tests of this component. I've provided the details of this into the ticket itself, so let's continue the conversation inside JIRA itself to avoid making noise here @ the user forum.
Sorry. I should have run the unit tests before submitting the patch. I've submitted an updated patch with changes to the unit test mock setup for the new Exchange copy behavior and the sessionID reversal in the MessagePredicate. I agree we can continue any patch-related discussion using Jira. However, the following discussion might be of general interest.
> And regarding your surprise maybe the following thread could be helpful for a better understanding:
>
> http://camel.465427.n5.nabble.com/About-what-to-do-with-the-Response-retrieved-through-a-Producer-when-the-Exchange-is-NOT-out-capable-td5713946.html
>
> In general if the MEP is InOnly and a service invocation (through a Camel Producer) returns a reply, then a good practice is to set the reply as the body of the IN message. This would make the next processor in chain to make use of this IN object as it say here:
This is confusing to me because the Exchange is now InOut, not InOnly. I did some experimentation and found that this route,
from("quickfix:examples/inprocess.cfg?sessionID=FIX.4.2:MARKET->TRADER&exchangePattern=InOut")
.filter(header(QuickfixjEndpoint.MESSAGE_TYPE_KEY).isEqualTo(MsgType.ORDER_STATUS_REQUEST))
.bean(new MarketOrderStatusService())
.bean(new QuickfixjMessageJsonPrinter());
required the Exchange copy for the InOut MEP because the inbound message is overwritten by subsequent processors. However,
the following route does not require the copy (the QuickfixjMessageJsonPrinter bean has been removed).
from("quickfix:examples/inprocess.cfg?sessionID=FIX.4.2:MARKET->TRADER&exchangePattern=InOut")
.filter(header(QuickfixjEndpoint.MESSAGE_TYPE_KEY).isEqualTo(MsgType.ORDER_STATUS_REQUEST))
.bean(new MarketOrderStatusService());
Does this mean that the QuickfixjConsumer is sometimes required to copy the inbound Exchange (InOut MEP) and sometimes not,
depending on the route components that follow it? If so, the safe behavior is to always copy the Exchange, but it seems that it should
logically not need to do that for an InOut MEP.
> if there is no out message then the in message is used instead
> http://camel.apache.org/using-getin-or-getout-methods-on-exchange.html
>
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728217.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi Steve
Thanks for looking into this. Unfortunately the provided patch causes
regressions by some pre-existing tests of this component. I've provided the
details of this into the ticket itself, so let's continue the conversation
inside JIRA itself to avoid making noise here @ the user forum.
And regarding your surprise maybe the following thread could be helpful for
a better understanding:
http://camel.465427.n5.nabble.com/About-what-to-do-with-the-Response-retrieved-through-a-Producer-when-the-Exchange-is-NOT-out-capable-td5713946.html
In general if the MEP is InOnly and a service invocation (through a Camel
Producer) returns a reply, then a good practice is to set the reply as the
body of the IN message. This would make the next processor in chain to make
use of this IN object as it say here:
> if there is no out message then the in message is used instead
http://camel.apache.org/using-getin-or-getout-methods-on-exchange.html
Babak
stevebate wrote
> On Feb 25, 2013, at 2:21 PM, Babak Vahdat [via Camel] <
> ml-node+s465427n5728116h57@.nabble
> > wrote:
>> As I did *not* backport the fix for CAMEL-5861 to this branch (because it
>> was/is not a supported version anymore). However the example works
>> properly on the 2.10.x branch as well as trunk with the proper JSON
>> outputs for both the request (OrderStatusRequest) as well as the reply
>> (ExecutionReport). But as because the QuickfixjConsumer doesn't honor the
>> MEP (apparently it used to honor this at the time you donated it) there
>> is currently no way to provide the ExecutionReport JSON output as the
>> HTTP reply itself. Please take a look at the log output I've added to the
>> example (the first comment of the ticket):
>>
>> https://issues.apache.org/jira/browse/CAMEL-5880
>>
>> As you see *although* we ask for the exchangePattern=InOut option through
>> the URI the logged message shows an effective InOnly MEP!
>
> Hello Babak,
>
> I've found several issues with the example. I have a patch that fixes the
> InOnly MEP issue. I also found an issue with the MessagePredicate. It was
> reversing the FIX Session ID so the MessageCorrelator wasn't working
> correctly. I was surprised to see that when the component processes an
> exchange (QuickfixjConsumer.onExchange), Camel overwrites the inbound
> message in the exchange with the result of the service invocation. I
> copied the inbound exchange and restored the "In" before sending the
> service response. Finally, I reverted the modification to the message
> correlation so that the ExecutionReport is used instead of the
> OrderStatusRequest (2.10.x branch).
>
> The IOException was being raised because the ExecutionReport result was
> never correlated to the OrderStatusRequest. This was because of the InOut
> MEP issue and the problem with the MessagePredicate that is used for the
> correlation. The HTTP request didn't have a result to return so that
> caused the 500 server error after the message correlation timed out.
>
> With my patch, the example appears to fully work. I'll attach it to the
> Jira issue for your review.
>
>> Please be aware that I've got no expertise about the QuickFixJ framework
>> or the FIX protocol itself and the fixes I've provided here are based on
>> a best effort basis to make things work again (no HTTP 500 anymore). But
>> having you as the founder of QuickFixJ and the Guru here in this thread
>> makes me really happy! So please take a look at the stuff here to see if
>> we could do even better.
>
> I have very little experience with Camel, so hopefully we can help each
> other on this task.
>
>> Last but not least that would be just awesome if we could make a proper
>> Camel example module out of these classes which are currently under the
>> path src/test/java of the camel-quickfix module itself, that's to make
>> them like the other examples we have:
>
> I agree, but I'm not sure if I'll have time to make this change. We'll
> see.
>
> Steve
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728204.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by Christian Müller <ch...@gmail.com>.
Thanks Steve! Very appreciated!
Best,
Christian
Sent from a mobile device
Am 26.02.2013 17:39 schrieb "stevebate" <sv...@stevebate.net>:
>
> On Feb 25, 2013, at 2:21 PM, Babak Vahdat [via Camel] <
> ml-node+s465427n5728116h57@n5.nabble.com> wrote:
> > As I did *not* backport the fix for CAMEL-5861 to this branch (because
> it was/is not a supported version anymore). However the example works
> properly on the 2.10.x branch as well as trunk with the proper JSON outputs
> for both the request (OrderStatusRequest) as well as the reply
> (ExecutionReport). But as because the QuickfixjConsumer doesn't honor the
> MEP (apparently it used to honor this at the time you donated it) there is
> currently no way to provide the ExecutionReport JSON output as the HTTP
> reply itself. Please take a look at the log output I've added to the
> example (the first comment of the ticket):
> >
> > https://issues.apache.org/jira/browse/CAMEL-5880
> >
> > As you see *although* we ask for the exchangePattern=InOut option
> through the URI the logged message shows an effective InOnly MEP!
>
> Hello Babak,
>
> I've found several issues with the example. I have a patch that fixes the
> InOnly MEP issue. I also found an issue with the MessagePredicate. It was
> reversing the FIX Session ID so the MessageCorrelator wasn't working
> correctly. I was surprised to see that when the component processes an
> exchange (QuickfixjConsumer.onExchange), Camel overwrites the inbound
> message in the exchange with the result of the service invocation. I copied
> the inbound exchange and restored the "In" before sending the service
> response. Finally, I reverted the modification to the message correlation
> so that the ExecutionReport is used instead of the OrderStatusRequest
> (2.10.x branch).
>
> The IOException was being raised because the ExecutionReport result was
> never correlated to the OrderStatusRequest. This was because of the InOut
> MEP issue and the problem with the MessagePredicate that is used for the
> correlation. The HTTP request didn't have a result to return so that caused
> the 500 server error after the message correlation timed out.
>
> With my patch, the example appears to fully work. I'll attach it to the
> Jira issue for your review.
>
> > Please be aware that I've got no expertise about the QuickFixJ framework
> or the FIX protocol itself and the fixes I've provided here are based on a
> best effort basis to make things work again (no HTTP 500 anymore). But
> having you as the founder of QuickFixJ and the Guru here in this thread
> makes me really happy! So please take a look at the stuff here to see if we
> could do even better.
>
> I have very little experience with Camel, so hopefully we can help each
> other on this task.
>
> > Last but not least that would be just awesome if we could make a proper
> Camel example module out of these classes which are currently under the
> path src/test/java of the camel-quickfix module itself, that's to make them
> like the other examples we have:
>
> I agree, but I'm not sure if I'll have time to make this change. We'll see.
>
> Steve
>
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728201.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by stevebate <sv...@stevebate.net>.
On Feb 25, 2013, at 2:21 PM, Babak Vahdat [via Camel] <ml...@n5.nabble.com> wrote:
> As I did *not* backport the fix for CAMEL-5861 to this branch (because it was/is not a supported version anymore). However the example works properly on the 2.10.x branch as well as trunk with the proper JSON outputs for both the request (OrderStatusRequest) as well as the reply (ExecutionReport). But as because the QuickfixjConsumer doesn't honor the MEP (apparently it used to honor this at the time you donated it) there is currently no way to provide the ExecutionReport JSON output as the HTTP reply itself. Please take a look at the log output I've added to the example (the first comment of the ticket):
>
> https://issues.apache.org/jira/browse/CAMEL-5880
>
> As you see *although* we ask for the exchangePattern=InOut option through the URI the logged message shows an effective InOnly MEP!
Hello Babak,
I've found several issues with the example. I have a patch that fixes the InOnly MEP issue. I also found an issue with the MessagePredicate. It was reversing the FIX Session ID so the MessageCorrelator wasn't working correctly. I was surprised to see that when the component processes an exchange (QuickfixjConsumer.onExchange), Camel overwrites the inbound message in the exchange with the result of the service invocation. I copied the inbound exchange and restored the "In" before sending the service response. Finally, I reverted the modification to the message correlation so that the ExecutionReport is used instead of the OrderStatusRequest (2.10.x branch).
The IOException was being raised because the ExecutionReport result was never correlated to the OrderStatusRequest. This was because of the InOut MEP issue and the problem with the MessagePredicate that is used for the correlation. The HTTP request didn't have a result to return so that caused the 500 server error after the message correlation timed out.
With my patch, the example appears to fully work. I'll attach it to the Jira issue for your review.
> Please be aware that I've got no expertise about the QuickFixJ framework or the FIX protocol itself and the fixes I've provided here are based on a best effort basis to make things work again (no HTTP 500 anymore). But having you as the founder of QuickFixJ and the Guru here in this thread makes me really happy! So please take a look at the stuff here to see if we could do even better.
I have very little experience with Camel, so hopefully we can help each other on this task.
> Last but not least that would be just awesome if we could make a proper Camel example module out of these classes which are currently under the path src/test/java of the camel-quickfix module itself, that's to make them like the other examples we have:
I agree, but I'm not sure if I'll have time to make this change. We'll see.
Steve
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728201.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel-quickfix RequestReplyExample java.io.IOException
Posted by Babak Vahdat <ba...@swissonline.ch>.
> I'll need to refresh my memory since the original code was written four
years
> ago.
> However, I'm sure the example worked at that time or I wouldn't have
> submitted or
> taken the time to document the example.
Hi
If you would run the example RequestReplyExample from the 2.8.x branch:
https://svn.apache.org/repos/asf/camel/branches/camel-2.8.x/
Then as lawkai has already reported by the first post in this thread this is
what you would get:
<20130225-18:17:20, FIX.4.2:MARKET->TRADER, event> (Session
FIX.4.2:MARKET->TRADER schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20130225-18:17:20, FIX.4.2:MARKET->TRADER, event> (Created session:
FIX.4.2:MARKET->TRADER)
<20130225-18:17:20, FIX.4.2:TRADER->MARKET, event> (Session
FIX.4.2:TRADER->MARKET schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20130225-18:17:20, FIX.4.2:TRADER->MARKET, event> (Created session:
FIX.4.2:TRADER->MARKET)
<20130225-18:17:21, FIX.4.2:TRADER->MARKET, outgoing>
(8=FIX.4.29=6835=A34=149=TRADER52=20130225-18:17:21.26656=MARKET98=0108=12010=185)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, incoming>
(8=FIX.4.29=6835=A34=149=TRADER52=20130225-18:17:21.26656=MARKET98=0108=12010=185)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, event> (Accepting session
FIX.4.2:MARKET->TRADER from anonymous(1671857013))
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, event> (Acceptor heartbeat set
to 120 seconds)
<20130225-18:17:21, FIX.4.2:TRADER->MARKET, event> (Initiated logon request)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, event> (Received logon)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, outgoing>
(8=FIX.4.29=6835=A34=149=MARKET52=20130225-18:17:21.28456=TRADER98=0108=12010=185)
<20130225-18:17:21, FIX.4.2:TRADER->MARKET, incoming>
(8=FIX.4.29=6835=A34=149=MARKET52=20130225-18:17:21.28456=TRADER98=0108=12010=185)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, event> (Responding to logon
request)
<20130225-18:17:21, FIX.4.2:TRADER->MARKET, event> (Received logon)
<20130225-18:17:21, FIX.4.2:TRADER->MARKET, outgoing>
(8=FIX.4.29=8235=H34=249=TRADER52=20130225-18:17:21.36056=MARKET11=XYZ37=abc54=155=GOOG10=148)
<20130225-18:17:21, FIX.4.2:MARKET->TRADER, incoming>
(8=FIX.4.29=8235=H34=249=TRADER52=20130225-18:17:21.36056=MARKET11=XYZ37=abc54=155=GOOG10=148)
Exception in thread "main" java.io.IOException: Server returned HTTP
response code: 500 for URL:
http://localhost:9123/order/status?sessionID=FIX.4.2:TRADER->MARKET&orderID=abc
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1615)
at
org.apache.camel.component.quickfixj.examples.RequestReplyExample.run(RequestReplyExample.java:106)
at
org.apache.camel.component.quickfixj.examples.RequestReplyExample.main(RequestReplyExample.java:63)
As I did *not* backport the fix for CAMEL-5861 to this branch (because it
was/is not a supported version anymore). However the example works properly
on the 2.10.x branch as well as trunk with the proper JSON outputs for both
the request (OrderStatusRequest) as well as the reply (ExecutionReport). But
as because the QuickfixjConsumer doesn't honor the MEP (apparently it used
to honor this at the time you donated it) there is currently no way to
provide the ExecutionReport JSON output as the HTTP reply itself. Please
take a look at the log output I've added to the example (the first comment
of the ticket):
https://issues.apache.org/jira/browse/CAMEL-5880
As you see *although* we ask for the exchangePattern=InOut option through
the URI the logged message shows an effective InOnly MEP!
Please be aware that I've got no expertise about the QuickFixJ framework or
the FIX protocol itself and the fixes I've provided here are based on a best
effort basis to make things work again (no HTTP 500 anymore). But having you
as the founder of QuickFixJ and the Guru here in this thread makes me really
happy! So please take a look at the stuff here to see if we could do even
better.
Last but not least that would be just awesome if we could make a proper
Camel example module out of these classes which are currently under the path
src/test/java of the camel-quickfix module itself, that's to make them like
the other examples we have:
http://camel.apache.org/examples.html
Because the users will mostly properly will not *even* see/realize these
camel-quickfix examples as they are under the path src/test/java which's
actually intended only for the unit-testing of the Maven module itself. Also
to be able to run them, currently one has to import them into his favorite
IDE first and then run them *manually* through the IDE itself. If we could
just provide "Maven runnable examples" out-of-the-box for these classes then
this would be great, e.g. using the exec-maven-plugin as we do here:
https://svn.apache.org/repos/asf/camel/trunk/examples/camel-example-twitter-websocket/README.txt
> When I have some time, I'll set up a Camel development environment to take
> a
> look at
> the issue. In the meantime, I have a few questions.
> You say that the QuickFIX/J component doesn't support the InOut MEP but
> there's a unit
> test to verify that it does (QuickfixjConsumerTest.processInOutExchange).
> I'm assuming
> that unit test is passing. Is that true?
Yes
> Looking at the SVN trunk, it appears that the RequestReplyExample is still
> incorrectly
> correlating the request and not the reply. It should be correlating with
> the
> ExecutionReport
> and not the OrderStatusRequest. Is that change reversed on a branch
> somewhere?
Regarding this reported issue both the trunk as well as the 2.10.x is in
sync as the commits have been backported.
Appreciate your help and engagement!
Babak
--
View this message in context: http://camel.465427.n5.nabble.com/camel-quickfix-RequestReplyExample-java-io-IOException-tp5723769p5728116.html
Sent from the Camel - Users mailing list archive at Nabble.com.