You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by rajan agarwal <ra...@gmail.com> on 2016/04/09 16:22:55 UTC
Calling external rest service multiple times in a loop
Hi All,
I am facing a issue when i try to call a external webservice in a loop. It
runs fine if the loop runs for upto 40 iterations but begins failing after
that. I am unable to understand the cause as well. I think that it might be
due to some sort of limit getting reached, i implemented a throttler as
well, but the result was same.
My loop is as below:
<loop>
<simple>${body[EndIndex]}</simple>
<setBody><simple>${null}</simple></setBody>
<setHeader headerName="Exchange.HTTP_URI">
<simple>
http://XX.XX.XX.XX:8080/rest/YY/${property.CamelLoopIndex}</simple>
</setHeader>
<setHeader headerName="CamelHttpMethod">
<constant>GET</constant>
</setHeader>
<to uri="http4://dummy"/>
<bean ref="in" method="adddataCache" />
<log message="body after GET cache ${body}" />
</loop>
Following is the stacktrace:
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.apache.camel.component.http4.HttpOperationFailedException: HTTP
operation failed invoking http4://XX.XX.XX.XX:8080/rest/YY/42 with
statusCode: 400
at
org.apache.camel.component.http4.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:236)[426:org.apache.camel.camel-http4:
2.15.1.redhat-620133]
at
org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:163)[426:org.apache.camel.camel-http4:2.15.1.redhat-620133]
at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[198:org.
apache.camel.camel-core:2.15.1.redhat-620133]
at
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
t-620133]
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
0133]
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
0133]
at
org.apache.camel.processor.DelayProcessorSupport.process(DelayProcessorSupport.java:160)[198:org.apache.camel.camel-core:2.15.1.redhat-6201
33]
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
t-620133]
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
0133]
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
0133]
at
org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
Any suggestions?
Re: Calling external rest service multiple times in a loop
Posted by Ioannis Mavroukakis <im...@gmail.com>.
Yes, but curl is hardly a yardstick to measure. For one, you're testing sequentially with curl, which means you close your connection before starting the next one. If we consider a server limit, bear in mind that http4 in Camel uses keep-alive. You can control that using the connectionTimeToLive parameter.
> On 9 Apr 2016, at 17:18, rajan agarwal <ra...@gmail.com> wrote:
>
> Hi Loannis,
>
>> On Apr 9, 2016 20:18, "Ioannis Mavroukakis" <im...@gmail.com> wrote:
>>
>> You're getting a 400 back, but you're not giving us a whole lot to go on
> :-).
> Its all I am getting :)
>>
>> Is there a limit set server side perhaps? Can you consistently sustain 39
> iterations?
> Yes It goes up to 41. I don't think there is a server side limit. I created
> a shell script to test that. I fire up to 100 using curl in the shell
> script and it works great without a sweat!
>>
>>> On 9 Apr 2016, at 15:22, rajan agarwal <ra...@gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> I am facing a issue when i try to call a external webservice in a loop.
> It
>>> runs fine if the loop runs for upto 40 iterations but begins failing
> after
>>> that. I am unable to understand the cause as well. I think that it
> might be
>>> due to some sort of limit getting reached, i implemented a throttler as
>>> well, but the result was same.
>>>
>>> My loop is as below:
>>>
>>> <loop>
>>> <simple>${body[EndIndex]}</simple>
>>> <setBody><simple>${null}</simple></setBody>
>>> <setHeader headerName="Exchange.HTTP_URI">
>>> <simple>
>>> http://XX.XX.XX.XX:8080/rest/YY/${property.CamelLoopIndex}</simple>
>>> </setHeader>
>>> <setHeader headerName="CamelHttpMethod">
>>> <constant>GET</constant>
>>> </setHeader>
>>> <to uri="http4://dummy"/>
>>>
>>> <bean ref="in" method="adddataCache" />
>>> <log message="body after GET cache ${body}" />
>>> </loop>
>>>
>>>
>>>
>>> Following is the stacktrace:
>>>
>>> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
>>> org.apache.camel.component.http4.HttpOperationFailedException: HTTP
>>> operation failed invoking http4://XX.XX.XX.XX:8080/rest/YY/42 with
>>> statusCode: 400
>>> at
> org.apache.camel.component.http4.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:236)[426:org.apache.camel.camel-http4:
>>> 2.15.1.redhat-620133]
>>> at
> org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:163)[426:org.apache.camel.camel-http4:2.15.1.redhat-620133]
>>> at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[198:org.
>>> apache.camel.camel-core:2.15.1.redhat-620133]
>>> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
>>> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
>>> t-620133]
>>> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
>>> 0133]
>>> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
>>> 0133]
>>> at
> org.apache.camel.processor.DelayProcessorSupport.process(DelayProcessorSupport.java:160)[198:org.apache.camel.camel-core:2.15.1.redhat-6201
>>> 33]
>>> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
>>> t-620133]
>>> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
>>> 0133]
>>> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
>>> 0133]
>>> at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
>>>
>>>
>>>
>>>
>>> Any suggestions?
Re: Calling external rest service multiple times in a loop
Posted by rajan agarwal <ra...@gmail.com>.
Hi Loannis,
On Apr 9, 2016 20:18, "Ioannis Mavroukakis" <im...@gmail.com> wrote:
>
> You're getting a 400 back, but you're not giving us a whole lot to go on
:-).
Its all I am getting :)
>
> Is there a limit set server side perhaps? Can you consistently sustain 39
iterations?
Yes It goes up to 41. I don't think there is a server side limit. I created
a shell script to test that. I fire up to 100 using curl in the shell
script and it works great without a sweat!
>
> > On 9 Apr 2016, at 15:22, rajan agarwal <ra...@gmail.com> wrote:
> >
> > Hi All,
> >
> > I am facing a issue when i try to call a external webservice in a loop.
It
> > runs fine if the loop runs for upto 40 iterations but begins failing
after
> > that. I am unable to understand the cause as well. I think that it
might be
> > due to some sort of limit getting reached, i implemented a throttler as
> > well, but the result was same.
> >
> > My loop is as below:
> >
> > <loop>
> > <simple>${body[EndIndex]}</simple>
> > <setBody><simple>${null}</simple></setBody>
> > <setHeader headerName="Exchange.HTTP_URI">
> > <simple>
> > http://XX.XX.XX.XX:8080/rest/YY/${property.CamelLoopIndex}</simple>
> > </setHeader>
> > <setHeader headerName="CamelHttpMethod">
> > <constant>GET</constant>
> > </setHeader>
> > <to uri="http4://dummy"/>
> >
> > <bean ref="in" method="adddataCache" />
> > <log message="body after GET cache ${body}" />
> > </loop>
> >
> >
> >
> > Following is the stacktrace:
> >
> > Stacktrace
> >
---------------------------------------------------------------------------------------------------------------------------------------
> > org.apache.camel.component.http4.HttpOperationFailedException: HTTP
> > operation failed invoking http4://XX.XX.XX.XX:8080/rest/YY/42 with
> > statusCode: 400
> > at
> >
org.apache.camel.component.http4.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:236)[426:org.apache.camel.camel-http4:
> > 2.15.1.redhat-620133]
> > at
> >
org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:163)[426:org.apache.camel.camel-http4:2.15.1.redhat-620133]
> > at
> >
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[198:org.
> > apache.camel.camel-core:2.15.1.redhat-620133]
> > at
> >
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
> > at
> >
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
> > t-620133]
> > at
> >
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> > 0133]
> > at
> >
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> > 0133]
> > at
> >
org.apache.camel.processor.DelayProcessorSupport.process(DelayProcessorSupport.java:160)[198:org.apache.camel.camel-core:2.15.1.redhat-6201
> > 33]
> > at
> >
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
> > t-620133]
> > at
> >
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> > 0133]
> > at
> >
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> > 0133]
> > at
> >
org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
> >
> >
> >
> >
> > Any suggestions?
Re: Calling external rest service multiple times in a loop
Posted by Ioannis Mavroukakis <im...@gmail.com>.
You're getting a 400 back, but you're not giving us a whole lot to go on :-).
Is there a limit set server side perhaps? Can you consistently sustain 39 iterations?
> On 9 Apr 2016, at 15:22, rajan agarwal <ra...@gmail.com> wrote:
>
> Hi All,
>
> I am facing a issue when i try to call a external webservice in a loop. It
> runs fine if the loop runs for upto 40 iterations but begins failing after
> that. I am unable to understand the cause as well. I think that it might be
> due to some sort of limit getting reached, i implemented a throttler as
> well, but the result was same.
>
> My loop is as below:
>
> <loop>
> <simple>${body[EndIndex]}</simple>
> <setBody><simple>${null}</simple></setBody>
> <setHeader headerName="Exchange.HTTP_URI">
> <simple>
> http://XX.XX.XX.XX:8080/rest/YY/${property.CamelLoopIndex}</simple>
> </setHeader>
> <setHeader headerName="CamelHttpMethod">
> <constant>GET</constant>
> </setHeader>
> <to uri="http4://dummy"/>
>
> <bean ref="in" method="adddataCache" />
> <log message="body after GET cache ${body}" />
> </loop>
>
>
>
> Following is the stacktrace:
>
> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
> org.apache.camel.component.http4.HttpOperationFailedException: HTTP
> operation failed invoking http4://XX.XX.XX.XX:8080/rest/YY/42 with
> statusCode: 400
> at
> org.apache.camel.component.http4.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:236)[426:org.apache.camel.camel-http4:
> 2.15.1.redhat-620133]
> at
> org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:163)[426:org.apache.camel.camel-http4:2.15.1.redhat-620133]
> at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[198:org.
> apache.camel.camel-core:2.15.1.redhat-620133]
> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
> t-620133]
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> 0133]
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> 0133]
> at
> org.apache.camel.processor.DelayProcessorSupport.process(DelayProcessorSupport.java:160)[198:org.apache.camel.camel-core:2.15.1.redhat-6201
> 33]
> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[198:org.apache.camel.camel-core:2.15.1.redha
> t-620133]
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> 0133]
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[198:org.apache.camel.camel-core:2.15.1.redhat-62
> 0133]
> at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[198:org.apache.camel.camel-core:2.15.1.redhat-620133]
>
>
>
>
> Any suggestions?