You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by rosmons <ro...@gmail.com> on 2011/03/16 01:34:34 UTC
Http route returning with 200 but also getting read timed out
Hi,
We have recently upgraded to Camel 2.5.0 from 1.4.0.
Almost everything seems to be working after making all the necessary
changes.
I'm attaching a route definition as a separate file.
In my route i am posting an xml message to a recipientList which maybe
http/ftp/sftp.
Now, everything seems to be working fine except that in the case of a
particular client route(http) I get 200 as the response code but I also seem
to get a Read timed out and onException block seems to be handling it and
passing to another route.
Now, the problem I find is that in this scenario the exchange that is passed
to the next route by the Error handler has the message body as null.
I'm attaching the log with level set to TRACE so you can see the trace logs.
I have checked the TRACE log for httpclient and it seems to be fine.
To me it looks like a bug in camel's TransactionErrorHandler but I'm not
able to find the exact cause.
Any help is appreciated.
thanks,
Rosmon
attached files:
http://camel.465427.n5.nabble.com/file/n3741631/camelRoute.txt
camelRoute.txt
http://camel.465427.n5.nabble.com/file/n3741631/CamelLog.txt CamelLog.txt
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3741631.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Willem Jiang <wi...@gmail.com>.
Hi Hari,
I can reproduce the error by writing a simple unit test.
I just create a JIRA[1] for it, will commit short fix for this case.
[1]https://issues.apache.org/jira/browse/CAMEL-3792
Willem
On 3/19/11 2:12 AM, Hari Gangadharan wrote:
> Willem:
>
> We have tried everything. We were already setting useOriginalMessage=true.
> Here is the route:
>
>
>
>
>
>
>
> java.lang.Exception
>
>
> true
>
>
>
>
>
> ChannelA
>
>
>
>
> $routerRoute
>
>
>
>
>
> We are getting messages in seda:routerDeliveryAttempt1Queue with null
> message body which is not acceptable. You need to look why the
> useOriginalMessage is ignored by Camel in this scenario. I can guarantee you
> that this is not the behavior of Camel 1.x. In Camel 1.x we didn't even have
> the useOriginalMessage.
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3975821.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Re: Http route returning with 200 but also getting read timed out
Posted by Hari Gangadharan <ha...@hotmail.com>.
Willem:
We have tried everything. We were already setting useOriginalMessage=true.
Here is the route:
java.lang.Exception
true
ChannelA
$routerRoute
We are getting messages in seda:routerDeliveryAttempt1Queue with null
message body which is not acceptable. You need to look why the
useOriginalMessage is ignored by Camel in this scenario. I can guarantee you
that this is not the behavior of Camel 1.x. In Camel 1.x we didn't even have
the useOriginalMessage.
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3975821.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Willem Jiang <wi...@gmail.com>.
Hi
I know what's you mean after checking out the patch that you provided.
Current camel http code just assume that there is a response, and create
a null message after exchange.getOut is called.
You change just make sure the exchange.getOut() will not be called if
the Timeout exception is thrown.
Now my concern is that the Http protocol is typical request-response
message. It could be strange , if we don't setup the out message for it.
Can I have a look at your route of handle error message ?
In Camel 2.x, you can setup the ErrorHandler to use the
UsingOriginalMessage option to make sure your failure processor can get
the original message.
Willem
On 3/18/11 2:29 AM, rosmons wrote:
> @Willem:
>
> We have found that this issue can be fixed by making a change to the
> HttpProducer.java class.
> The issue is that we're extracting the body after calling the
> "exchange.getOut()".
>
> This means that if there's an exception while extracting the body the
> original message will not get sent to the errorhandler/onException. Instead
> a message with empty headers and body is sent.
>
> I moved the "extractResponseBody(method, exchange)" to a line above
> "exchange.getOut()" and am able to get the desired result.
>
> You can see the attached diff file to see the change that we have made.
>
> http://camel.465427.n5.nabble.com/file/n3893573/HttpProducer.java.diff
> HttpProducer.java.diff
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3893573.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Re: Http route returning with 200 but also getting read timed out
Posted by rosmons <ro...@gmail.com>.
@Willem:
We have found that this issue can be fixed by making a change to the
HttpProducer.java class.
The issue is that we're extracting the body after calling the
"exchange.getOut()".
This means that if there's an exception while extracting the body the
original message will not get sent to the errorhandler/onException. Instead
a message with empty headers and body is sent.
I moved the "extractResponseBody(method, exchange)" to a line above
"exchange.getOut()" and am able to get the desired result.
You can see the attached diff file to see the change that we have made.
http://camel.465427.n5.nabble.com/file/n3893573/HttpProducer.java.diff
HttpProducer.java.diff
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3893573.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Willem Jiang <wi...@gmail.com>.
On 3/18/11 3:00 AM, Hari Gangadharan wrote:
> @Ashwin:
>
> We have looked at that consumer - they are too slow and it is the consumer's
> fault. Even if we increase the timeout to 20 sec, there would be another
> consumer that will have the same issue. My spec is to wait only 10 sec. I
> will try to redeliver until I am sure that it is processed successfully in
> the customer side. I understand that the HTTP connections are designed to
> behave this way. However I am concerned that Camel is throwing an exception
> *and* also processing the output of the producer - this may be an anomaly.
> The older versions of Camel behaved differently on read timeout. In Camel
> 1.x, in a read timeout, the Exception is thrown and the input message to the
> producer is passed through the pipeline. Now there is a difference: An
> exception is thrown and the output of the HTTP Producer is passed through
> the pipeline.
I'm assuming that you do not setup the error handler to redelivery the
message. Not sure why the out message could be used for further
processing, unless it is send to failure processor or dead letter processor.
>
> You can look at Rosmon's diff - if we make a small modification, Camel is
> working exactly like it used to work in 1.x. Anyone else have comments on
> what Camel should be doing on a read timeout??? According to me if an
> exception happened within a process then the process is not complete and we
> should not pass forward the half cooked output of the process. Isn't that a
> fair assumption?
It make sense, but I still want to have a look at your route.
>
> Hari Gangadharan
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3895320.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Re: Http route returning with 200 but also getting read timed out
Posted by Ashwin Karpe <ak...@fusesource.com>.
Add a Jira entry in Camel. Please see link below
https://issues.apache.org/jira/browse/CAMEL-3788
Cheers,
Ashwin...
-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
---------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3898320.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Ashwin Karpe <ak...@fusesource.com>.
Ah ok... Yep this behavior may be a bug.
I will try and create a Jira issue for this problem and link it back to this
issue for you to track and us to correct. BTW, can you please send me the
exact error that is being thrown. I will also check out version 1.0 to see
what was different.
Cheers,
Ashwin...
-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
---------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3897410.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Hari Gangadharan <ha...@hotmail.com>.
@Ashwin:
We have looked at that consumer - they are too slow and it is the consumer's
fault. Even if we increase the timeout to 20 sec, there would be another
consumer that will have the same issue. My spec is to wait only 10 sec. I
will try to redeliver until I am sure that it is processed successfully in
the customer side. I understand that the HTTP connections are designed to
behave this way. However I am concerned that Camel is throwing an exception
*and* also processing the output of the producer - this may be an anomaly.
The older versions of Camel behaved differently on read timeout. In Camel
1.x, in a read timeout, the Exception is thrown and the input message to the
producer is passed through the pipeline. Now there is a difference: An
exception is thrown and the output of the HTTP Producer is passed through
the pipeline.
You can look at Rosmon's diff - if we make a small modification, Camel is
working exactly like it used to work in 1.x. Anyone else have comments on
what Camel should be doing on a read timeout??? According to me if an
exception happened within a process then the process is not complete and we
should not pass forward the half cooked output of the process. Isn't that a
fair assumption?
Hari Gangadharan
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3895320.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Ashwin Karpe <ak...@fusesource.com>.
Hi,
Have you tried increasing the producers timeout window and introducing
keep-alives on the connection
Can you add the option httpClient.soTimeout=20000 to your HTTP Producer and
see if the producer hangs around for 20 seconds.
BTW, the fact that the HTTP client bails after a fixed period for a HTTP
Connection timeout is not a bug in Camel. This is how HTTP connections are
designed to behave. The best you can do is to set the connection timeout to
a higher value.
Also not sure what you are trying to do here, but trying to re-deliver a
message to a slow consumer can cause duplicate message sent to the consumer
and have the opposite effect where he get loaded now with redundant
requests. If the consumer is just slow and did not get back in time for the
connection window to be closed does not mean he did not process the message.
You will need to further configure your HTTP Producer to deal with the
scenario at hand. If on the other hand it is a true exception other than a
timeout, there is obviously no point in retrying.
Hope this helps.
Cheers,
Ashwin...
-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
---------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3893697.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Hari Gangadharan <ha...@hotmail.com>.
@Willem:
We will not be able to make server respond faster since it is customer's
server. I believe there is a bug in Camel framework. In this scenario, the
HTTP Producer thinks that the exchange has completed successfully whereas
the Camel framework thinks that the exchange has failed. Camel framework
puts this for redelivery. The end result is that the message which is passed
on is not the original message but the message output from the HTTP Producer
(which is an empty message).
The unfortunate thing is after a few redelivery attempts we use dead letter
processing to move it to a "ReDeliveryAttempt1" queue. Another pipeline
tries to redeliver this. However when we get this particular error our
message was replaced by a blank message.
Does this make sense?
Hari Gangadharan
--
View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3791254.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Http route returning with 200 but also getting read timed out
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
From the log I can see the http client get the 200 http response, but
the connection is closed after 10 seconds
Just checked the HttpProducer code it will call
InputStream is = method.getResponseBodyAsStream();
to get the response, it looks like the server can't return any data in
the 10 seconds.
Can you make sure the server return the response rightly ?
Willem
On 3/16/11 8:34 AM, rosmons wrote:
> Hi,
> We have recently upgraded to Camel 2.5.0 from 1.4.0.
> Almost everything seems to be working after making all the necessary
> changes.
> I'm attaching a route definition as a separate file.
> In my route i am posting an xml message to a recipientList which maybe
> http/ftp/sftp.
> Now, everything seems to be working fine except that in the case of a
> particular client route(http) I get 200 as the response code but I also seem
> to get a Read timed out and onException block seems to be handling it and
> passing to another route.
>
> Now, the problem I find is that in this scenario the exchange that is passed
> to the next route by the Error handler has the message body as null.
> I'm attaching the log with level set to TRACE so you can see the trace logs.
>
> I have checked the TRACE log for httpclient and it seems to be fine.
> To me it looks like a bug in camel's TransactionErrorHandler but I'm not
> able to find the exact cause.
> Any help is appreciated.
> thanks,
> Rosmon
>
>
> attached files:
> http://camel.465427.n5.nabble.com/file/n3741631/camelRoute.txt
> camelRoute.txt
> http://camel.465427.n5.nabble.com/file/n3741631/CamelLog.txt CamelLog.txt
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Http-route-returning-with-200-but-also-getting-read-timed-out-tp3741631p3741631.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang