You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by paquettd <da...@lmco.com> on 2009/03/02 17:54:48 UTC
Performance and MessageSupport.getBody (1.6.0)
I've been seeing some performance problems with Camel 1.6.0 (I have not tried
this with previous versions yet).
My profiler is pointing the finger at MessageSupport.getBody,
TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
specifically
findTypeConverter is always throwing a NoTypeConversionAvailableException;
which is then being caught and ignored in MessageSupport.getBody; at which
point processing continues successfully.
protected <T> T getBody(Class<T> type, Object body) is the specific getBody
in question.
Is this exception an expected behavior? It's weird how the catch block
doesn't even log a warning. Should a converter have been found? My message
payload is just a java.lang.String.
I suspect I've done something wrong but I don't know where to start looking.
I'm concerned with this; as I'm comparing Camel to some other message
routing solutions. This is making Camel take 40 times longer than the
competition and I want to make sure I do a fair comparison.
--
View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Mar 2, 2009 at 6:43 PM, paquettd <da...@lmco.com> wrote:
>
> I'm not sure if it makes a difference but I'm not using JMS anywhere. In fact
> in this test everything is using "direct".
>
> Is there something I can do in the Spring DSL to hint to Camel that there is
> no conversion necessary?
Can you show your route and what you send to the direct endpoint?
>
>
> Claus Ibsen-2 wrote:
>>
>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com> wrote:
>>>
>>> I've been seeing some performance problems with Camel 1.6.0 (I have not
>>> tried
>>> this with previous versions yet).
>>>
>>> My profiler is pointing the finger at MessageSupport.getBody,
>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>> specifically
>>>
>>> findTypeConverter is always throwing a
>>> NoTypeConversionAvailableException;
>>> which is then being caught and ignored in MessageSupport.getBody; at
>>> which
>>> point processing continues successfully.
>> This should be normal in situations where you ask the body to be a
>> specific type which cannot be converted to.
>> To help this can you show your route and what kind of JMS messages are
>> you sending.
>>
>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>> you can send whatever objects you like.
>> It has a rich type converter registry to be able to convert seamless
>> between types.
>>
>> This registry is loaded on demand, so you should make sure your start
>> profiling after Camel is "warm".
>>
>>
>>
>>>
>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>> getBody
>>> in question.
>>>
>>> Is this exception an expected behavior? It's weird how the catch block
>>> doesn't even log a warning. Should a converter have been found? My
>>> message
>>> payload is just a java.lang.String.
>> In the old days it returned null, but that did not work as the payload
>> you were trying to convert could be null, so it was a catch-22
>> situation.
>> So we added the exception.
>>
>> But if throwing exception is expensive we could maybe add a has test
>> to avoid this exception being thrown and caught in the MessageSupport.
>>
>> The exception is also meant for end users so they get a good exception
>> detailing the problem if they try to convert something into eg,
>> MyFooClass and its not possible to convert to it. Instead of returning
>> a null value as result.
>>
>>
>>>
>>> I suspect I've done something wrong but I don't know where to start
>>> looking.
>>> I'm concerned with this; as I'm comparing Camel to some other message
>>> routing solutions. This is making Camel take 40 times longer than the
>>> competition and I want to make sure I do a fair comparison.
>> We are currently rewamping the internal API in Camel 2.0 that leads us
>> up to a point where we can do performance improvements when Camel
>> routes exchanges.
>> Currently it does a bit of defensive copying when it moves message
>> from node to node. The revamped API lets us do some more clever stuff
>> there to improve the speed.
>>
>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>> then of course pure JMS to JMS is faster than eg over Camel as its a
>> very flexible and transport/protocol agnostic framework. But
>> performance improvements is on our roadmap in 2.1.
>>
>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by paquettd <da...@lmco.com>.
Good news; the 1.6.1-SNAPSHOT release had a big performance impact.
In my benchmarking the mean execution of the test case went from 8.5ms to
around 1ms. This is a great improvement!
I did a little work with the profiler running and I didn't see the issue
with the Exceptions anymore. There are still millions of calls to some of
these methods (over the span of 1000 messages) but they are much faster.
Thank you for following up on this issue.
Claus Ibsen-2 wrote:
>
> Hi Dan
>
> Thank you very much for reporting and doing the detective work of
> digging into the codebase and using a probe to detect the hot spot.
>
> I am about to commit a fix that avoids the excessive overhead of the
> NoTypeConverterException.
>
> In a simple test sending 1000 messages to a route, I get about 2x
> times the performance gain.
> Going from about 9 sec to 4,5 sec.
>
> This issue was caused by a StreamCacheInterceptor that was needed for
> better integration with ServiceMix.
> It will inspect the payloads for streams that are cache able and this
> that is why its asking for the payload to be converted to a
> StreamCache.
>
> I added a rude unit test to the camel code to output the times
> org.apache.camel.processor.RoutePerformanceTest
>
> The ticket is:
> https://issues.apache.org/activemq/browse/CAMEL-1417
>
> You are welcome to try your test again with a new SNAPSHOT of Camel
> 2.0 or 1.6.1 when I get it committed.
> We would be interested to hear your feedback and results.
>
>
>
> On Wed, Mar 4, 2009 at 9:24 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> I am looking into this now. Its an issue introduced by the StreamCache
>> eg:
>> No type converter available to convert from type: java.lang.Integer to
>> the required type: org.apache.camel.StreamCache with value 1
>>
>> Will create a ticket and get onto it
>>
>> On Tue, Mar 3, 2009 at 6:04 PM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>>> Hi
>>>
>>> Did we see which type it tries to convert to? You have only said it
>>> throws 10 exceptions but it should log which type it wanted to convert
>>> it to.
>>> That will also help.
>>>
>>>
>>> On Tue, Mar 3, 2009 at 4:35 PM, Claus Ibsen <cl...@gmail.com>
>>> wrote:
>>>> On Tue, Mar 3, 2009 at 4:19 PM, paquettd <da...@lmco.com> wrote:
>>>>>
>>>>> Not only avoiding the try..catch but just running through all the
>>>>> reflection
>>>>> code over and over again can't be cheap either.
>>>> It has a local cache so the lookup will be fast as its a map.
>>>>
>>>> But its the throws .. catch exception that is expensive if it does it
>>>> 10 times for your test case.
>>>> We gotta check into this. Maybe there is something obvious wrong.
>>>>
>>>> We can later improve it to not thrown the excpetion in certain
>>>> situations where we dont care.
>>>>
>>>>
>>>>>
>>>>>
>>>>> willem.jiang wrote:
>>>>>>
>>>>>> Maybe we could add some kind of type converter cache to avoid the
>>>>>> try
>>>>>> ... catch work to improve the performance.
>>>>>>
>>>>>> Willem
>>>>>>
>>>>>> paquettd wrote:
>>>>>>> Thank you for trying to help. I'm attaching my config file here. As
>>>>>>> you
>>>>>>> can
>>>>>>> see this is a contrived example I made to test performance (I used a
>>>>>>> similar
>>>>>>> example with some other products).
>>>>>>>
>>>>>>> Essentially what's happening is a String is being sent with the
>>>>>>> ProducerTemplate send method; setting the type to InOut. The string
>>>>>>> is
>>>>>>> checked for lower case characters; and if it has any it goes to the
>>>>>>> "capitalizer"; after that it goes to a reverser and truncator before
>>>>>>> finally
>>>>>>> a timestamp is put into the header (before I sent it I put a
>>>>>>> timestamp as
>>>>>>> well). I run this a few thousand times; using System.nanoTime for
>>>>>>> those
>>>>>>> timestamps (while not real accurate I used the same thing with other
>>>>>>> COTS
>>>>>>> and the relative differences are in time are larger than the clock
>>>>>>> granularity)
>>>>>>>
>>>>>>> After I started seeing long processing times I used JProbe from
>>>>>>> Quest
>>>>>>> Software to narrow down to the bottleneck we are talking about.
>>>>>>> JProbe
>>>>>>> tells
>>>>>>> me that Exception is thrown 10 times processing one of these
>>>>>>> messages....
>>>>>>> so
>>>>>>> when I send 1000 messages 10,000 of those exceptions are being
>>>>>>> thrown and
>>>>>>> it
>>>>>>> takes a very long time for the 1000 messages to get through.
>>>>>>>
>>>>>>>
>>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>>
>>>>>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>>>
>>>>>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>>>>>> http://activemq.apache.org/camel/schema/spring
>>>>>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>>>>>
>>>>>>> <bean id="reverser" class="camel.test.StringReverser" />
>>>>>>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>>>>>>> <bean id="truncater" class="camel.test.StringTruncater">
>>>>>>> <property name="length" value="10" />
>>>>>>> </bean>
>>>>>>> <bean id="endTimestamper"
>>>>>>> class="camel.test.transformers.HeaderTimestamper">
>>>>>>> <property name="headerKey" value="END_TIME"/>
>>>>>>> </bean>
>>>>>>>
>>>>>>> <camelContext id="camel.test.camelcontext"
>>>>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>>>> <template id="myCamelTemplate"
>>>>>>> defaultEndpoint="direct:test" />
>>>>>>>
>>>>>>> <route>
>>>>>>> <from uri="direct:test" />
>>>>>>> <choice>
>>>>>>> <when>
>>>>>>> <simple>${body} regex
>>>>>>> '.*[a-x].*'</simple>
>>>>>>> <to
>>>>>>> uri="direct:capitalizeFirst" />
>>>>>>> </when>
>>>>>>> <otherwise>
>>>>>>> <to
>>>>>>> uri="direct:reverseAndTruncate" />
>>>>>>> </otherwise>
>>>>>>> </choice>
>>>>>>> </route>
>>>>>>>
>>>>>>> <route>
>>>>>>> <from uri="direct:capitalizeFirst" />
>>>>>>> <bean ref="capitalizer" />
>>>>>>> <to uri="direct:reverseAndTruncate"/>
>>>>>>> </route>
>>>>>>>
>>>>>>> <route>
>>>>>>> <from uri="direct:reverseAndTruncate" />
>>>>>>> <bean ref="reverser" />
>>>>>>> <bean ref="truncater" />
>>>>>>> <to
>>>>>>> uri="bean:endTimestamper?method=timestamp"/>
>>>>>>> </route>
>>>>>>> </camelContext>
>>>>>>>
>>>>>>> <bean id="testDriver" class="camel.test.TestDriver">
>>>>>>> <property name="producerTemplate" ref="myCamelTemplate"
>>>>>>> />
>>>>>>> </bean>
>>>>>>> </beans>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang
>>>>>>>> <wi...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> If you don't want the TypeConverter to get invovled , you could
>>>>>>>>> use
>>>>>>>>> MessageSupport.getBody() directly.
>>>>>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we
>>>>>>>> need
>>>>>>>> to see this.
>>>>>>>>
>>>>>>>> Or some other endpoint/producer trying to get the body as a special
>>>>>>>> type, eg InputStream etc.
>>>>>>>>
>>>>>>>> But the route would really help.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Willem
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I'm not sure if it makes a difference but I'm not using JMS
>>>>>>>>>> anywhere.
>>>>>>>>>> In
>>>>>>>>>> fact
>>>>>>>>>> in this test everything is using "direct".
>>>>>>>>>>
>>>>>>>>>> Is there something I can do in the Spring DSL to hint to Camel
>>>>>>>>>> that
>>>>>>>>>> there
>>>>>>>>>> is
>>>>>>>>>> no conversion necessary?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I
>>>>>>>>>>>> have
>>>>>>>>>> not
>>>>>>>>>>>> tried
>>>>>>>>>>>> this with previous versions yet).
>>>>>>>>>>>>
>>>>>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>>>>>> TypeConverter.convertTo, and
>>>>>>>>>>>> DefaultTypeConverter.findTypeConverter
>>>>>>>>>>>> specifically
>>>>>>>>>>>>
>>>>>>>>>>>> findTypeConverter is always throwing a
>>>>>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>>>>>> which is then being caught and ignored in
>>>>>>>>>>>> MessageSupport.getBody; at
>>>>>>>>>>>> which
>>>>>>>>>>>> point processing continues successfully.
>>>>>>>>>>> This should be normal in situations where you ask the body to be
>>>>>>>>>>> a
>>>>>>>>>>> specific type which cannot be converted to.
>>>>>>>>>>> To help this can you show your route and what kind of JMS
>>>>>>>>>>> messages
>>>>>>>>>>> are
>>>>>>>>>>> you sending.
>>>>>>>>>>>
>>>>>>>>>>> Camel is payload type agnostics (eg dont have to be pure XML
>>>>>>>>>>> etc.) so
>>>>>>>>>>> you can send whatever objects you like.
>>>>>>>>>>> It has a rich type converter registry to be able to convert
>>>>>>>>>>> seamless
>>>>>>>>>>> between types.
>>>>>>>>>>>
>>>>>>>>>>> This registry is loaded on demand, so you should make sure your
>>>>>>>>>>> start
>>>>>>>>>>> profiling after Camel is "warm".
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the
>>>>>>>>>>>> specific
>>>>>>>>>>>> getBody
>>>>>>>>>>>> in question.
>>>>>>>>>>>>
>>>>>>>>>>>> Is this exception an expected behavior? It's weird how the
>>>>>>>>>>>> catch
>>>>>>>>>> block
>>>>>>>>>>>> doesn't even log a warning. Should a converter have been found?
>>>>>>>>>>>> My
>>>>>>>>>>>> message
>>>>>>>>>>>> payload is just a java.lang.String.
>>>>>>>>>>> In the old days it returned null, but that did not work as the
>>>>>>>>>>> payload
>>>>>>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>>>>>>> situation.
>>>>>>>>>>> So we added the exception.
>>>>>>>>>>>
>>>>>>>>>>> But if throwing exception is expensive we could maybe add a has
>>>>>>>>>>> test
>>>>>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>>>>>> MessageSupport.
>>>>>>>>>>>
>>>>>>>>>>> The exception is also meant for end users so they get a good
>>>>>>>>>>> exception
>>>>>>>>>>> detailing the problem if they try to convert something into eg,
>>>>>>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>>>>>>> returning
>>>>>>>>>>> a null value as result.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I suspect I've done something wrong but I don't know where to
>>>>>>>>>>>> start
>>>>>>>>>>>> looking.
>>>>>>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>>>>>>> message
>>>>>>>>>>>> routing solutions. This is making Camel take 40 times longer
>>>>>>>>>>>> than
>>>>>>>>>>>> the
>>>>>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>>>>>> We are currently rewamping the internal API in Camel 2.0 that
>>>>>>>>>>> leads
>>>>>>>>>>> us
>>>>>>>>>>> up to a point where we can do performance improvements when
>>>>>>>>>>> Camel
>>>>>>>>>>> routes exchanges.
>>>>>>>>>>> Currently it does a bit of defensive copying when it moves
>>>>>>>>>>> message
>>>>>>>>>>> from node to node. The revamped API lets us do some more clever
>>>>>>>>>>> stuff
>>>>>>>>>>> there to improve the speed.
>>>>>>>>>>>
>>>>>>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really
>>>>>>>>>>> fast
>>>>>>>>>>> then of course pure JMS to JMS is faster than eg over Camel as
>>>>>>>>>>> its a
>>>>>>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>
>>>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>
>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
>
--
View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22353744.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Dan
Thank you very much for reporting and doing the detective work of
digging into the codebase and using a probe to detect the hot spot.
I am about to commit a fix that avoids the excessive overhead of the
NoTypeConverterException.
In a simple test sending 1000 messages to a route, I get about 2x
times the performance gain.
Going from about 9 sec to 4,5 sec.
This issue was caused by a StreamCacheInterceptor that was needed for
better integration with ServiceMix.
It will inspect the payloads for streams that are cache able and this
that is why its asking for the payload to be converted to a
StreamCache.
I added a rude unit test to the camel code to output the times
org.apache.camel.processor.RoutePerformanceTest
The ticket is:
https://issues.apache.org/activemq/browse/CAMEL-1417
You are welcome to try your test again with a new SNAPSHOT of Camel
2.0 or 1.6.1 when I get it committed.
We would be interested to hear your feedback and results.
On Wed, Mar 4, 2009 at 9:24 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> I am looking into this now. Its an issue introduced by the StreamCache
> eg:
> No type converter available to convert from type: java.lang.Integer to
> the required type: org.apache.camel.StreamCache with value 1
>
> Will create a ticket and get onto it
>
> On Tue, Mar 3, 2009 at 6:04 PM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> Did we see which type it tries to convert to? You have only said it
>> throws 10 exceptions but it should log which type it wanted to convert
>> it to.
>> That will also help.
>>
>>
>> On Tue, Mar 3, 2009 at 4:35 PM, Claus Ibsen <cl...@gmail.com> wrote:
>>> On Tue, Mar 3, 2009 at 4:19 PM, paquettd <da...@lmco.com> wrote:
>>>>
>>>> Not only avoiding the try..catch but just running through all the reflection
>>>> code over and over again can't be cheap either.
>>> It has a local cache so the lookup will be fast as its a map.
>>>
>>> But its the throws .. catch exception that is expensive if it does it
>>> 10 times for your test case.
>>> We gotta check into this. Maybe there is something obvious wrong.
>>>
>>> We can later improve it to not thrown the excpetion in certain
>>> situations where we dont care.
>>>
>>>
>>>>
>>>>
>>>> willem.jiang wrote:
>>>>>
>>>>> Maybe we could add some kind of type converter cache to avoid the try
>>>>> ... catch work to improve the performance.
>>>>>
>>>>> Willem
>>>>>
>>>>> paquettd wrote:
>>>>>> Thank you for trying to help. I'm attaching my config file here. As you
>>>>>> can
>>>>>> see this is a contrived example I made to test performance (I used a
>>>>>> similar
>>>>>> example with some other products).
>>>>>>
>>>>>> Essentially what's happening is a String is being sent with the
>>>>>> ProducerTemplate send method; setting the type to InOut. The string is
>>>>>> checked for lower case characters; and if it has any it goes to the
>>>>>> "capitalizer"; after that it goes to a reverser and truncator before
>>>>>> finally
>>>>>> a timestamp is put into the header (before I sent it I put a timestamp as
>>>>>> well). I run this a few thousand times; using System.nanoTime for those
>>>>>> timestamps (while not real accurate I used the same thing with other COTS
>>>>>> and the relative differences are in time are larger than the clock
>>>>>> granularity)
>>>>>>
>>>>>> After I started seeing long processing times I used JProbe from Quest
>>>>>> Software to narrow down to the bottleneck we are talking about. JProbe
>>>>>> tells
>>>>>> me that Exception is thrown 10 times processing one of these messages....
>>>>>> so
>>>>>> when I send 1000 messages 10,000 of those exceptions are being thrown and
>>>>>> it
>>>>>> takes a very long time for the 1000 messages to get through.
>>>>>>
>>>>>>
>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>
>>>>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>>
>>>>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>>>>> http://activemq.apache.org/camel/schema/spring
>>>>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>>>>
>>>>>> <bean id="reverser" class="camel.test.StringReverser" />
>>>>>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>>>>>> <bean id="truncater" class="camel.test.StringTruncater">
>>>>>> <property name="length" value="10" />
>>>>>> </bean>
>>>>>> <bean id="endTimestamper"
>>>>>> class="camel.test.transformers.HeaderTimestamper">
>>>>>> <property name="headerKey" value="END_TIME"/>
>>>>>> </bean>
>>>>>>
>>>>>> <camelContext id="camel.test.camelcontext"
>>>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>>> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>>>>>>
>>>>>> <route>
>>>>>> <from uri="direct:test" />
>>>>>> <choice>
>>>>>> <when>
>>>>>> <simple>${body} regex '.*[a-x].*'</simple>
>>>>>> <to uri="direct:capitalizeFirst" />
>>>>>> </when>
>>>>>> <otherwise>
>>>>>> <to uri="direct:reverseAndTruncate" />
>>>>>> </otherwise>
>>>>>> </choice>
>>>>>> </route>
>>>>>>
>>>>>> <route>
>>>>>> <from uri="direct:capitalizeFirst" />
>>>>>> <bean ref="capitalizer" />
>>>>>> <to uri="direct:reverseAndTruncate"/>
>>>>>> </route>
>>>>>>
>>>>>> <route>
>>>>>> <from uri="direct:reverseAndTruncate" />
>>>>>> <bean ref="reverser" />
>>>>>> <bean ref="truncater" />
>>>>>> <to uri="bean:endTimestamper?method=timestamp"/>
>>>>>> </route>
>>>>>> </camelContext>
>>>>>>
>>>>>> <bean id="testDriver" class="camel.test.TestDriver">
>>>>>> <property name="producerTemplate" ref="myCamelTemplate" />
>>>>>> </bean>
>>>>>> </beans>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>>>>>>> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> If you don't want the TypeConverter to get invovled , you could use
>>>>>>>> MessageSupport.getBody() directly.
>>>>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>>>>>>> to see this.
>>>>>>>
>>>>>>> Or some other endpoint/producer trying to get the body as a special
>>>>>>> type, eg InputStream etc.
>>>>>>>
>>>>>>> But the route would really help.
>>>>>>>
>>>>>>>
>>>>>>>> Willem
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>>>>>>
>>>>>>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere.
>>>>>>>>> In
>>>>>>>>> fact
>>>>>>>>> in this test everything is using "direct".
>>>>>>>>>
>>>>>>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>>>>>>> there
>>>>>>>>> is
>>>>>>>>> no conversion necessary?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>>>>>> wrote:
>>>>>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>>>>>>> not
>>>>>>>>>>> tried
>>>>>>>>>>> this with previous versions yet).
>>>>>>>>>>>
>>>>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>>>>>> specifically
>>>>>>>>>>>
>>>>>>>>>>> findTypeConverter is always throwing a
>>>>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>>>>>>> which
>>>>>>>>>>> point processing continues successfully.
>>>>>>>>>> This should be normal in situations where you ask the body to be a
>>>>>>>>>> specific type which cannot be converted to.
>>>>>>>>>> To help this can you show your route and what kind of JMS messages
>>>>>>>>>> are
>>>>>>>>>> you sending.
>>>>>>>>>>
>>>>>>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>>>>>>> you can send whatever objects you like.
>>>>>>>>>> It has a rich type converter registry to be able to convert seamless
>>>>>>>>>> between types.
>>>>>>>>>>
>>>>>>>>>> This registry is loaded on demand, so you should make sure your start
>>>>>>>>>> profiling after Camel is "warm".
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>>>>>>> getBody
>>>>>>>>>>> in question.
>>>>>>>>>>>
>>>>>>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>>>>>>> block
>>>>>>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>>>>>>> message
>>>>>>>>>>> payload is just a java.lang.String.
>>>>>>>>>> In the old days it returned null, but that did not work as the
>>>>>>>>>> payload
>>>>>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>>>>>> situation.
>>>>>>>>>> So we added the exception.
>>>>>>>>>>
>>>>>>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>>>>> MessageSupport.
>>>>>>>>>>
>>>>>>>>>> The exception is also meant for end users so they get a good
>>>>>>>>>> exception
>>>>>>>>>> detailing the problem if they try to convert something into eg,
>>>>>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>>>>>> returning
>>>>>>>>>> a null value as result.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>>>>>>> looking.
>>>>>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>>>>>> message
>>>>>>>>>>> routing solutions. This is making Camel take 40 times longer than
>>>>>>>>>>> the
>>>>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>>>>> We are currently rewamping the internal API in Camel 2.0 that leads
>>>>>>>>>> us
>>>>>>>>>> up to a point where we can do performance improvements when Camel
>>>>>>>>>> routes exchanges.
>>>>>>>>>> Currently it does a bit of defensive copying when it moves message
>>>>>>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>>>>>>> there to improve the speed.
>>>>>>>>>>
>>>>>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>>>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>>
>>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I am looking into this now. Its an issue introduced by the StreamCache
eg:
No type converter available to convert from type: java.lang.Integer to
the required type: org.apache.camel.StreamCache with value 1
Will create a ticket and get onto it
On Tue, Mar 3, 2009 at 6:04 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Did we see which type it tries to convert to? You have only said it
> throws 10 exceptions but it should log which type it wanted to convert
> it to.
> That will also help.
>
>
> On Tue, Mar 3, 2009 at 4:35 PM, Claus Ibsen <cl...@gmail.com> wrote:
>> On Tue, Mar 3, 2009 at 4:19 PM, paquettd <da...@lmco.com> wrote:
>>>
>>> Not only avoiding the try..catch but just running through all the reflection
>>> code over and over again can't be cheap either.
>> It has a local cache so the lookup will be fast as its a map.
>>
>> But its the throws .. catch exception that is expensive if it does it
>> 10 times for your test case.
>> We gotta check into this. Maybe there is something obvious wrong.
>>
>> We can later improve it to not thrown the excpetion in certain
>> situations where we dont care.
>>
>>
>>>
>>>
>>> willem.jiang wrote:
>>>>
>>>> Maybe we could add some kind of type converter cache to avoid the try
>>>> ... catch work to improve the performance.
>>>>
>>>> Willem
>>>>
>>>> paquettd wrote:
>>>>> Thank you for trying to help. I'm attaching my config file here. As you
>>>>> can
>>>>> see this is a contrived example I made to test performance (I used a
>>>>> similar
>>>>> example with some other products).
>>>>>
>>>>> Essentially what's happening is a String is being sent with the
>>>>> ProducerTemplate send method; setting the type to InOut. The string is
>>>>> checked for lower case characters; and if it has any it goes to the
>>>>> "capitalizer"; after that it goes to a reverser and truncator before
>>>>> finally
>>>>> a timestamp is put into the header (before I sent it I put a timestamp as
>>>>> well). I run this a few thousand times; using System.nanoTime for those
>>>>> timestamps (while not real accurate I used the same thing with other COTS
>>>>> and the relative differences are in time are larger than the clock
>>>>> granularity)
>>>>>
>>>>> After I started seeing long processing times I used JProbe from Quest
>>>>> Software to narrow down to the bottleneck we are talking about. JProbe
>>>>> tells
>>>>> me that Exception is thrown 10 times processing one of these messages....
>>>>> so
>>>>> when I send 1000 messages 10,000 of those exceptions are being thrown and
>>>>> it
>>>>> takes a very long time for the 1000 messages to get through.
>>>>>
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>
>>>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>
>>>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>>>> http://activemq.apache.org/camel/schema/spring
>>>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>>>
>>>>> <bean id="reverser" class="camel.test.StringReverser" />
>>>>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>>>>> <bean id="truncater" class="camel.test.StringTruncater">
>>>>> <property name="length" value="10" />
>>>>> </bean>
>>>>> <bean id="endTimestamper"
>>>>> class="camel.test.transformers.HeaderTimestamper">
>>>>> <property name="headerKey" value="END_TIME"/>
>>>>> </bean>
>>>>>
>>>>> <camelContext id="camel.test.camelcontext"
>>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>>>>>
>>>>> <route>
>>>>> <from uri="direct:test" />
>>>>> <choice>
>>>>> <when>
>>>>> <simple>${body} regex '.*[a-x].*'</simple>
>>>>> <to uri="direct:capitalizeFirst" />
>>>>> </when>
>>>>> <otherwise>
>>>>> <to uri="direct:reverseAndTruncate" />
>>>>> </otherwise>
>>>>> </choice>
>>>>> </route>
>>>>>
>>>>> <route>
>>>>> <from uri="direct:capitalizeFirst" />
>>>>> <bean ref="capitalizer" />
>>>>> <to uri="direct:reverseAndTruncate"/>
>>>>> </route>
>>>>>
>>>>> <route>
>>>>> <from uri="direct:reverseAndTruncate" />
>>>>> <bean ref="reverser" />
>>>>> <bean ref="truncater" />
>>>>> <to uri="bean:endTimestamper?method=timestamp"/>
>>>>> </route>
>>>>> </camelContext>
>>>>>
>>>>> <bean id="testDriver" class="camel.test.TestDriver">
>>>>> <property name="producerTemplate" ref="myCamelTemplate" />
>>>>> </bean>
>>>>> </beans>
>>>>>
>>>>>
>>>>>
>>>>> Claus Ibsen-2 wrote:
>>>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>>>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> If you don't want the TypeConverter to get invovled , you could use
>>>>>>> MessageSupport.getBody() directly.
>>>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>>>>>> to see this.
>>>>>>
>>>>>> Or some other endpoint/producer trying to get the body as a special
>>>>>> type, eg InputStream etc.
>>>>>>
>>>>>> But the route would really help.
>>>>>>
>>>>>>
>>>>>>> Willem
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>>>>>
>>>>>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere.
>>>>>>>> In
>>>>>>>> fact
>>>>>>>> in this test everything is using "direct".
>>>>>>>>
>>>>>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>>>>>> there
>>>>>>>> is
>>>>>>>> no conversion necessary?
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>>>>> wrote:
>>>>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>>>>>> not
>>>>>>>>>> tried
>>>>>>>>>> this with previous versions yet).
>>>>>>>>>>
>>>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>>>>> specifically
>>>>>>>>>>
>>>>>>>>>> findTypeConverter is always throwing a
>>>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>>>>>> which
>>>>>>>>>> point processing continues successfully.
>>>>>>>>> This should be normal in situations where you ask the body to be a
>>>>>>>>> specific type which cannot be converted to.
>>>>>>>>> To help this can you show your route and what kind of JMS messages
>>>>>>>>> are
>>>>>>>>> you sending.
>>>>>>>>>
>>>>>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>>>>>> you can send whatever objects you like.
>>>>>>>>> It has a rich type converter registry to be able to convert seamless
>>>>>>>>> between types.
>>>>>>>>>
>>>>>>>>> This registry is loaded on demand, so you should make sure your start
>>>>>>>>> profiling after Camel is "warm".
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>>>>>> getBody
>>>>>>>>>> in question.
>>>>>>>>>>
>>>>>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>>>>>> block
>>>>>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>>>>>> message
>>>>>>>>>> payload is just a java.lang.String.
>>>>>>>>> In the old days it returned null, but that did not work as the
>>>>>>>>> payload
>>>>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>>>>> situation.
>>>>>>>>> So we added the exception.
>>>>>>>>>
>>>>>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>>>> MessageSupport.
>>>>>>>>>
>>>>>>>>> The exception is also meant for end users so they get a good
>>>>>>>>> exception
>>>>>>>>> detailing the problem if they try to convert something into eg,
>>>>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>>>>> returning
>>>>>>>>> a null value as result.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>>>>>> looking.
>>>>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>>>>> message
>>>>>>>>>> routing solutions. This is making Camel take 40 times longer than
>>>>>>>>>> the
>>>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>>>> We are currently rewamping the internal API in Camel 2.0 that leads
>>>>>>>>> us
>>>>>>>>> up to a point where we can do performance improvements when Camel
>>>>>>>>> routes exchanges.
>>>>>>>>> Currently it does a bit of defensive copying when it moves message
>>>>>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>>>>>> there to improve the speed.
>>>>>>>>>
>>>>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>>
>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Did we see which type it tries to convert to? You have only said it
throws 10 exceptions but it should log which type it wanted to convert
it to.
That will also help.
On Tue, Mar 3, 2009 at 4:35 PM, Claus Ibsen <cl...@gmail.com> wrote:
> On Tue, Mar 3, 2009 at 4:19 PM, paquettd <da...@lmco.com> wrote:
>>
>> Not only avoiding the try..catch but just running through all the reflection
>> code over and over again can't be cheap either.
> It has a local cache so the lookup will be fast as its a map.
>
> But its the throws .. catch exception that is expensive if it does it
> 10 times for your test case.
> We gotta check into this. Maybe there is something obvious wrong.
>
> We can later improve it to not thrown the excpetion in certain
> situations where we dont care.
>
>
>>
>>
>> willem.jiang wrote:
>>>
>>> Maybe we could add some kind of type converter cache to avoid the try
>>> ... catch work to improve the performance.
>>>
>>> Willem
>>>
>>> paquettd wrote:
>>>> Thank you for trying to help. I'm attaching my config file here. As you
>>>> can
>>>> see this is a contrived example I made to test performance (I used a
>>>> similar
>>>> example with some other products).
>>>>
>>>> Essentially what's happening is a String is being sent with the
>>>> ProducerTemplate send method; setting the type to InOut. The string is
>>>> checked for lower case characters; and if it has any it goes to the
>>>> "capitalizer"; after that it goes to a reverser and truncator before
>>>> finally
>>>> a timestamp is put into the header (before I sent it I put a timestamp as
>>>> well). I run this a few thousand times; using System.nanoTime for those
>>>> timestamps (while not real accurate I used the same thing with other COTS
>>>> and the relative differences are in time are larger than the clock
>>>> granularity)
>>>>
>>>> After I started seeing long processing times I used JProbe from Quest
>>>> Software to narrow down to the bottleneck we are talking about. JProbe
>>>> tells
>>>> me that Exception is thrown 10 times processing one of these messages....
>>>> so
>>>> when I send 1000 messages 10,000 of those exceptions are being thrown and
>>>> it
>>>> takes a very long time for the 1000 messages to get through.
>>>>
>>>>
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>
>>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>
>>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>>> http://activemq.apache.org/camel/schema/spring
>>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>>
>>>> <bean id="reverser" class="camel.test.StringReverser" />
>>>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>>>> <bean id="truncater" class="camel.test.StringTruncater">
>>>> <property name="length" value="10" />
>>>> </bean>
>>>> <bean id="endTimestamper"
>>>> class="camel.test.transformers.HeaderTimestamper">
>>>> <property name="headerKey" value="END_TIME"/>
>>>> </bean>
>>>>
>>>> <camelContext id="camel.test.camelcontext"
>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>>>>
>>>> <route>
>>>> <from uri="direct:test" />
>>>> <choice>
>>>> <when>
>>>> <simple>${body} regex '.*[a-x].*'</simple>
>>>> <to uri="direct:capitalizeFirst" />
>>>> </when>
>>>> <otherwise>
>>>> <to uri="direct:reverseAndTruncate" />
>>>> </otherwise>
>>>> </choice>
>>>> </route>
>>>>
>>>> <route>
>>>> <from uri="direct:capitalizeFirst" />
>>>> <bean ref="capitalizer" />
>>>> <to uri="direct:reverseAndTruncate"/>
>>>> </route>
>>>>
>>>> <route>
>>>> <from uri="direct:reverseAndTruncate" />
>>>> <bean ref="reverser" />
>>>> <bean ref="truncater" />
>>>> <to uri="bean:endTimestamper?method=timestamp"/>
>>>> </route>
>>>> </camelContext>
>>>>
>>>> <bean id="testDriver" class="camel.test.TestDriver">
>>>> <property name="producerTemplate" ref="myCamelTemplate" />
>>>> </bean>
>>>> </beans>
>>>>
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>>>>> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> If you don't want the TypeConverter to get invovled , you could use
>>>>>> MessageSupport.getBody() directly.
>>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>>>>> to see this.
>>>>>
>>>>> Or some other endpoint/producer trying to get the body as a special
>>>>> type, eg InputStream etc.
>>>>>
>>>>> But the route would really help.
>>>>>
>>>>>
>>>>>> Willem
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>>>>
>>>>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere.
>>>>>>> In
>>>>>>> fact
>>>>>>> in this test everything is using "direct".
>>>>>>>
>>>>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>>>>> there
>>>>>>> is
>>>>>>> no conversion necessary?
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>>>> wrote:
>>>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>>>>> not
>>>>>>>>> tried
>>>>>>>>> this with previous versions yet).
>>>>>>>>>
>>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>>>> specifically
>>>>>>>>>
>>>>>>>>> findTypeConverter is always throwing a
>>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>>>>> which
>>>>>>>>> point processing continues successfully.
>>>>>>>> This should be normal in situations where you ask the body to be a
>>>>>>>> specific type which cannot be converted to.
>>>>>>>> To help this can you show your route and what kind of JMS messages
>>>>>>>> are
>>>>>>>> you sending.
>>>>>>>>
>>>>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>>>>> you can send whatever objects you like.
>>>>>>>> It has a rich type converter registry to be able to convert seamless
>>>>>>>> between types.
>>>>>>>>
>>>>>>>> This registry is loaded on demand, so you should make sure your start
>>>>>>>> profiling after Camel is "warm".
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>>>>> getBody
>>>>>>>>> in question.
>>>>>>>>>
>>>>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>>>>> block
>>>>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>>>>> message
>>>>>>>>> payload is just a java.lang.String.
>>>>>>>> In the old days it returned null, but that did not work as the
>>>>>>>> payload
>>>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>>>> situation.
>>>>>>>> So we added the exception.
>>>>>>>>
>>>>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>>> MessageSupport.
>>>>>>>>
>>>>>>>> The exception is also meant for end users so they get a good
>>>>>>>> exception
>>>>>>>> detailing the problem if they try to convert something into eg,
>>>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>>>> returning
>>>>>>>> a null value as result.
>>>>>>>>
>>>>>>>>
>>>>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>>>>> looking.
>>>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>>>> message
>>>>>>>>> routing solutions. This is making Camel take 40 times longer than
>>>>>>>>> the
>>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>>> We are currently rewamping the internal API in Camel 2.0 that leads
>>>>>>>> us
>>>>>>>> up to a point where we can do performance improvements when Camel
>>>>>>>> routes exchanges.
>>>>>>>> Currently it does a bit of defensive copying when it moves message
>>>>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>>>>> there to improve the speed.
>>>>>>>>
>>>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>>
>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>>
>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>> --
>> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Mar 3, 2009 at 4:19 PM, paquettd <da...@lmco.com> wrote:
>
> Not only avoiding the try..catch but just running through all the reflection
> code over and over again can't be cheap either.
It has a local cache so the lookup will be fast as its a map.
But its the throws .. catch exception that is expensive if it does it
10 times for your test case.
We gotta check into this. Maybe there is something obvious wrong.
We can later improve it to not thrown the excpetion in certain
situations where we dont care.
>
>
> willem.jiang wrote:
>>
>> Maybe we could add some kind of type converter cache to avoid the try
>> ... catch work to improve the performance.
>>
>> Willem
>>
>> paquettd wrote:
>>> Thank you for trying to help. I'm attaching my config file here. As you
>>> can
>>> see this is a contrived example I made to test performance (I used a
>>> similar
>>> example with some other products).
>>>
>>> Essentially what's happening is a String is being sent with the
>>> ProducerTemplate send method; setting the type to InOut. The string is
>>> checked for lower case characters; and if it has any it goes to the
>>> "capitalizer"; after that it goes to a reverser and truncator before
>>> finally
>>> a timestamp is put into the header (before I sent it I put a timestamp as
>>> well). I run this a few thousand times; using System.nanoTime for those
>>> timestamps (while not real accurate I used the same thing with other COTS
>>> and the relative differences are in time are larger than the clock
>>> granularity)
>>>
>>> After I started seeing long processing times I used JProbe from Quest
>>> Software to narrow down to the bottleneck we are talking about. JProbe
>>> tells
>>> me that Exception is thrown 10 times processing one of these messages....
>>> so
>>> when I send 1000 messages 10,000 of those exceptions are being thrown and
>>> it
>>> takes a very long time for the 1000 messages to get through.
>>>
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>
>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>> http://activemq.apache.org/camel/schema/spring
>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>
>>> <bean id="reverser" class="camel.test.StringReverser" />
>>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>>> <bean id="truncater" class="camel.test.StringTruncater">
>>> <property name="length" value="10" />
>>> </bean>
>>> <bean id="endTimestamper"
>>> class="camel.test.transformers.HeaderTimestamper">
>>> <property name="headerKey" value="END_TIME"/>
>>> </bean>
>>>
>>> <camelContext id="camel.test.camelcontext"
>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>>>
>>> <route>
>>> <from uri="direct:test" />
>>> <choice>
>>> <when>
>>> <simple>${body} regex '.*[a-x].*'</simple>
>>> <to uri="direct:capitalizeFirst" />
>>> </when>
>>> <otherwise>
>>> <to uri="direct:reverseAndTruncate" />
>>> </otherwise>
>>> </choice>
>>> </route>
>>>
>>> <route>
>>> <from uri="direct:capitalizeFirst" />
>>> <bean ref="capitalizer" />
>>> <to uri="direct:reverseAndTruncate"/>
>>> </route>
>>>
>>> <route>
>>> <from uri="direct:reverseAndTruncate" />
>>> <bean ref="reverser" />
>>> <bean ref="truncater" />
>>> <to uri="bean:endTimestamper?method=timestamp"/>
>>> </route>
>>> </camelContext>
>>>
>>> <bean id="testDriver" class="camel.test.TestDriver">
>>> <property name="producerTemplate" ref="myCamelTemplate" />
>>> </bean>
>>> </beans>
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> If you don't want the TypeConverter to get invovled , you could use
>>>>> MessageSupport.getBody() directly.
>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>>>> to see this.
>>>>
>>>> Or some other endpoint/producer trying to get the body as a special
>>>> type, eg InputStream etc.
>>>>
>>>> But the route would really help.
>>>>
>>>>
>>>>> Willem
>>>>>
>>>>>
>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>>>
>>>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere.
>>>>>> In
>>>>>> fact
>>>>>> in this test everything is using "direct".
>>>>>>
>>>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>>>> there
>>>>>> is
>>>>>> no conversion necessary?
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>>> wrote:
>>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>>>> not
>>>>>>>> tried
>>>>>>>> this with previous versions yet).
>>>>>>>>
>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>>> specifically
>>>>>>>>
>>>>>>>> findTypeConverter is always throwing a
>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>>>> which
>>>>>>>> point processing continues successfully.
>>>>>>> This should be normal in situations where you ask the body to be a
>>>>>>> specific type which cannot be converted to.
>>>>>>> To help this can you show your route and what kind of JMS messages
>>>>>>> are
>>>>>>> you sending.
>>>>>>>
>>>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>>>> you can send whatever objects you like.
>>>>>>> It has a rich type converter registry to be able to convert seamless
>>>>>>> between types.
>>>>>>>
>>>>>>> This registry is loaded on demand, so you should make sure your start
>>>>>>> profiling after Camel is "warm".
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>>>> getBody
>>>>>>>> in question.
>>>>>>>>
>>>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>>>> block
>>>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>>>> message
>>>>>>>> payload is just a java.lang.String.
>>>>>>> In the old days it returned null, but that did not work as the
>>>>>>> payload
>>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>>> situation.
>>>>>>> So we added the exception.
>>>>>>>
>>>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>> MessageSupport.
>>>>>>>
>>>>>>> The exception is also meant for end users so they get a good
>>>>>>> exception
>>>>>>> detailing the problem if they try to convert something into eg,
>>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>>> returning
>>>>>>> a null value as result.
>>>>>>>
>>>>>>>
>>>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>>>> looking.
>>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>>> message
>>>>>>>> routing solutions. This is making Camel take 40 times longer than
>>>>>>>> the
>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>> We are currently rewamping the internal API in Camel 2.0 that leads
>>>>>>> us
>>>>>>> up to a point where we can do performance improvements when Camel
>>>>>>> routes exchanges.
>>>>>>> Currently it does a bit of defensive copying when it moves message
>>>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>>>> there to improve the speed.
>>>>>>>
>>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>
>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>>
>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>>
>>>>
>>>
>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by paquettd <da...@lmco.com>.
Not only avoiding the try..catch but just running through all the reflection
code over and over again can't be cheap either.
willem.jiang wrote:
>
> Maybe we could add some kind of type converter cache to avoid the try
> ... catch work to improve the performance.
>
> Willem
>
> paquettd wrote:
>> Thank you for trying to help. I'm attaching my config file here. As you
>> can
>> see this is a contrived example I made to test performance (I used a
>> similar
>> example with some other products).
>>
>> Essentially what's happening is a String is being sent with the
>> ProducerTemplate send method; setting the type to InOut. The string is
>> checked for lower case characters; and if it has any it goes to the
>> "capitalizer"; after that it goes to a reverser and truncator before
>> finally
>> a timestamp is put into the header (before I sent it I put a timestamp as
>> well). I run this a few thousand times; using System.nanoTime for those
>> timestamps (while not real accurate I used the same thing with other COTS
>> and the relative differences are in time are larger than the clock
>> granularity)
>>
>> After I started seeing long processing times I used JProbe from Quest
>> Software to narrow down to the bottleneck we are talking about. JProbe
>> tells
>> me that Exception is thrown 10 times processing one of these messages....
>> so
>> when I send 1000 messages 10,000 of those exceptions are being thrown and
>> it
>> takes a very long time for the 1000 messages to get through.
>>
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <beans xmlns="http://www.springframework.org/schema/beans"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>
>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>
>> http://www.springframework.org/schema/beans/spring-beans.xsd
>> http://activemq.apache.org/camel/schema/spring
>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>
>> <bean id="reverser" class="camel.test.StringReverser" />
>> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
>> <bean id="truncater" class="camel.test.StringTruncater">
>> <property name="length" value="10" />
>> </bean>
>> <bean id="endTimestamper"
>> class="camel.test.transformers.HeaderTimestamper">
>> <property name="headerKey" value="END_TIME"/>
>> </bean>
>>
>> <camelContext id="camel.test.camelcontext"
>> xmlns="http://activemq.apache.org/camel/schema/spring">
>> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>>
>> <route>
>> <from uri="direct:test" />
>> <choice>
>> <when>
>> <simple>${body} regex '.*[a-x].*'</simple>
>> <to uri="direct:capitalizeFirst" />
>> </when>
>> <otherwise>
>> <to uri="direct:reverseAndTruncate" />
>> </otherwise>
>> </choice>
>> </route>
>>
>> <route>
>> <from uri="direct:capitalizeFirst" />
>> <bean ref="capitalizer" />
>> <to uri="direct:reverseAndTruncate"/>
>> </route>
>>
>> <route>
>> <from uri="direct:reverseAndTruncate" />
>> <bean ref="reverser" />
>> <bean ref="truncater" />
>> <to uri="bean:endTimestamper?method=timestamp"/>
>> </route>
>> </camelContext>
>>
>> <bean id="testDriver" class="camel.test.TestDriver">
>> <property name="producerTemplate" ref="myCamelTemplate" />
>> </bean>
>> </beans>
>>
>>
>>
>> Claus Ibsen-2 wrote:
>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>>> wrote:
>>>> Hi,
>>>>
>>>> If you don't want the TypeConverter to get invovled , you could use
>>>> MessageSupport.getBody() directly.
>>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>>> to see this.
>>>
>>> Or some other endpoint/producer trying to get the body as a special
>>> type, eg InputStream etc.
>>>
>>> But the route would really help.
>>>
>>>
>>>> Willem
>>>>
>>>>
>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>>
>>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere.
>>>>> In
>>>>> fact
>>>>> in this test everything is using "direct".
>>>>>
>>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>>> there
>>>>> is
>>>>> no conversion necessary?
>>>>>
>>>>>
>>>>> Claus Ibsen-2 wrote:
>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>>> wrote:
>>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>>> not
>>>>>>> tried
>>>>>>> this with previous versions yet).
>>>>>>>
>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>> specifically
>>>>>>>
>>>>>>> findTypeConverter is always throwing a
>>>>>>> NoTypeConversionAvailableException;
>>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>>> which
>>>>>>> point processing continues successfully.
>>>>>> This should be normal in situations where you ask the body to be a
>>>>>> specific type which cannot be converted to.
>>>>>> To help this can you show your route and what kind of JMS messages
>>>>>> are
>>>>>> you sending.
>>>>>>
>>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>>> you can send whatever objects you like.
>>>>>> It has a rich type converter registry to be able to convert seamless
>>>>>> between types.
>>>>>>
>>>>>> This registry is loaded on demand, so you should make sure your start
>>>>>> profiling after Camel is "warm".
>>>>>>
>>>>>>
>>>>>>
>>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>>> getBody
>>>>>>> in question.
>>>>>>>
>>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>>> block
>>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>>> message
>>>>>>> payload is just a java.lang.String.
>>>>>> In the old days it returned null, but that did not work as the
>>>>>> payload
>>>>>> you were trying to convert could be null, so it was a catch-22
>>>>>> situation.
>>>>>> So we added the exception.
>>>>>>
>>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>>> to avoid this exception being thrown and caught in the
>>>>>> MessageSupport.
>>>>>>
>>>>>> The exception is also meant for end users so they get a good
>>>>>> exception
>>>>>> detailing the problem if they try to convert something into eg,
>>>>>> MyFooClass and its not possible to convert to it. Instead of
>>>>>> returning
>>>>>> a null value as result.
>>>>>>
>>>>>>
>>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>>> looking.
>>>>>>> I'm concerned with this; as I'm comparing Camel to some other
>>>>>>> message
>>>>>>> routing solutions. This is making Camel take 40 times longer than
>>>>>>> the
>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>> We are currently rewamping the internal API in Camel 2.0 that leads
>>>>>> us
>>>>>> up to a point where we can do performance improvements when Camel
>>>>>> routes exchanges.
>>>>>> Currently it does a bit of defensive copying when it moves message
>>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>>> there to improve the speed.
>>>>>>
>>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>>> very flexible and transport/protocol agnostic framework. But
>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>
>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>>
>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>
>>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>>
>>>
>>
>
>
>
--
View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Willem Jiang <wi...@gmail.com>.
Maybe we could add some kind of type converter cache to avoid the try
... catch work to improve the performance.
Willem
paquettd wrote:
> Thank you for trying to help. I'm attaching my config file here. As you can
> see this is a contrived example I made to test performance (I used a similar
> example with some other products).
>
> Essentially what's happening is a String is being sent with the
> ProducerTemplate send method; setting the type to InOut. The string is
> checked for lower case characters; and if it has any it goes to the
> "capitalizer"; after that it goes to a reverser and truncator before finally
> a timestamp is put into the header (before I sent it I put a timestamp as
> well). I run this a few thousand times; using System.nanoTime for those
> timestamps (while not real accurate I used the same thing with other COTS
> and the relative differences are in time are larger than the clock
> granularity)
>
> After I started seeing long processing times I used JProbe from Quest
> Software to narrow down to the bottleneck we are talking about. JProbe tells
> me that Exception is thrown 10 times processing one of these messages.... so
> when I send 1000 messages 10,000 of those exceptions are being thrown and it
> takes a very long time for the 1000 messages to get through.
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
>
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://activemq.apache.org/camel/schema/spring
> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>
> <bean id="reverser" class="camel.test.StringReverser" />
> <bean id="capitalizer" class="camel.test.StringCapitalizer" />
> <bean id="truncater" class="camel.test.StringTruncater">
> <property name="length" value="10" />
> </bean>
> <bean id="endTimestamper"
> class="camel.test.transformers.HeaderTimestamper">
> <property name="headerKey" value="END_TIME"/>
> </bean>
>
> <camelContext id="camel.test.camelcontext"
> xmlns="http://activemq.apache.org/camel/schema/spring">
> <template id="myCamelTemplate" defaultEndpoint="direct:test" />
>
> <route>
> <from uri="direct:test" />
> <choice>
> <when>
> <simple>${body} regex '.*[a-x].*'</simple>
> <to uri="direct:capitalizeFirst" />
> </when>
> <otherwise>
> <to uri="direct:reverseAndTruncate" />
> </otherwise>
> </choice>
> </route>
>
> <route>
> <from uri="direct:capitalizeFirst" />
> <bean ref="capitalizer" />
> <to uri="direct:reverseAndTruncate"/>
> </route>
>
> <route>
> <from uri="direct:reverseAndTruncate" />
> <bean ref="reverser" />
> <bean ref="truncater" />
> <to uri="bean:endTimestamper?method=timestamp"/>
> </route>
> </camelContext>
>
> <bean id="testDriver" class="camel.test.TestDriver">
> <property name="producerTemplate" ref="myCamelTemplate" />
> </bean>
> </beans>
>
>
>
> Claus Ibsen-2 wrote:
>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
>> wrote:
>>> Hi,
>>>
>>> If you don't want the TypeConverter to get invovled , you could use
>>> MessageSupport.getBody() directly.
>> Yes I am wondering if he has a .convertBodyTo in the route, so we need
>> to see this.
>>
>> Or some other endpoint/producer trying to get the body as a special
>> type, eg InputStream etc.
>>
>> But the route would really help.
>>
>>
>>> Willem
>>>
>>>
>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>>
>>>> I'm not sure if it makes a difference but I'm not using JMS anywhere. In
>>>> fact
>>>> in this test everything is using "direct".
>>>>
>>>> Is there something I can do in the Spring DSL to hint to Camel that
>>>> there
>>>> is
>>>> no conversion necessary?
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>>> wrote:
>>>>>> I've been seeing some performance problems with Camel 1.6.0 (I have
>>>> not
>>>>>> tried
>>>>>> this with previous versions yet).
>>>>>>
>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>> specifically
>>>>>>
>>>>>> findTypeConverter is always throwing a
>>>>>> NoTypeConversionAvailableException;
>>>>>> which is then being caught and ignored in MessageSupport.getBody; at
>>>>>> which
>>>>>> point processing continues successfully.
>>>>> This should be normal in situations where you ask the body to be a
>>>>> specific type which cannot be converted to.
>>>>> To help this can you show your route and what kind of JMS messages are
>>>>> you sending.
>>>>>
>>>>> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>>>> you can send whatever objects you like.
>>>>> It has a rich type converter registry to be able to convert seamless
>>>>> between types.
>>>>>
>>>>> This registry is loaded on demand, so you should make sure your start
>>>>> profiling after Camel is "warm".
>>>>>
>>>>>
>>>>>
>>>>>> protected <T> T getBody(Class<T> type, Object body) is the specific
>>>>>> getBody
>>>>>> in question.
>>>>>>
>>>>>> Is this exception an expected behavior? It's weird how the catch
>>>> block
>>>>>> doesn't even log a warning. Should a converter have been found? My
>>>>>> message
>>>>>> payload is just a java.lang.String.
>>>>> In the old days it returned null, but that did not work as the payload
>>>>> you were trying to convert could be null, so it was a catch-22
>>>>> situation.
>>>>> So we added the exception.
>>>>>
>>>>> But if throwing exception is expensive we could maybe add a has test
>>>>> to avoid this exception being thrown and caught in the MessageSupport.
>>>>>
>>>>> The exception is also meant for end users so they get a good exception
>>>>> detailing the problem if they try to convert something into eg,
>>>>> MyFooClass and its not possible to convert to it. Instead of returning
>>>>> a null value as result.
>>>>>
>>>>>
>>>>>> I suspect I've done something wrong but I don't know where to start
>>>>>> looking.
>>>>>> I'm concerned with this; as I'm comparing Camel to some other message
>>>>>> routing solutions. This is making Camel take 40 times longer than the
>>>>>> competition and I want to make sure I do a fair comparison.
>>>>> We are currently rewamping the internal API in Camel 2.0 that leads us
>>>>> up to a point where we can do performance improvements when Camel
>>>>> routes exchanges.
>>>>> Currently it does a bit of defensive copying when it moves message
>>>>> from node to node. The revamped API lets us do some more clever stuff
>>>>> there to improve the speed.
>>>>>
>>>>> So if you are testing, eg. JMS -> JMS and want it to be really fast
>>>>> then of course pure JMS to JMS is faster than eg over Camel as its a
>>>>> very flexible and transport/protocol agnostic framework. But
>>>>> performance improvements is on our roadmap in 2.1.
>>>>>
>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>
>>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>>
>
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by paquettd <da...@lmco.com>.
Thank you for trying to help. I'm attaching my config file here. As you can
see this is a contrived example I made to test performance (I used a similar
example with some other products).
Essentially what's happening is a String is being sent with the
ProducerTemplate send method; setting the type to InOut. The string is
checked for lower case characters; and if it has any it goes to the
"capitalizer"; after that it goes to a reverser and truncator before finally
a timestamp is put into the header (before I sent it I put a timestamp as
well). I run this a few thousand times; using System.nanoTime for those
timestamps (while not real accurate I used the same thing with other COTS
and the relative differences are in time are larger than the clock
granularity)
After I started seeing long processing times I used JProbe from Quest
Software to narrow down to the bottleneck we are talking about. JProbe tells
me that Exception is thrown 10 times processing one of these messages.... so
when I send 1000 messages 10,000 of those exceptions are being thrown and it
takes a very long time for the 1000 messages to get through.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://activemq.apache.org/camel/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
<bean id="reverser" class="camel.test.StringReverser" />
<bean id="capitalizer" class="camel.test.StringCapitalizer" />
<bean id="truncater" class="camel.test.StringTruncater">
<property name="length" value="10" />
</bean>
<bean id="endTimestamper"
class="camel.test.transformers.HeaderTimestamper">
<property name="headerKey" value="END_TIME"/>
</bean>
<camelContext id="camel.test.camelcontext"
xmlns="http://activemq.apache.org/camel/schema/spring">
<template id="myCamelTemplate" defaultEndpoint="direct:test" />
<route>
<from uri="direct:test" />
<choice>
<when>
<simple>${body} regex '.*[a-x].*'</simple>
<to uri="direct:capitalizeFirst" />
</when>
<otherwise>
<to uri="direct:reverseAndTruncate" />
</otherwise>
</choice>
</route>
<route>
<from uri="direct:capitalizeFirst" />
<bean ref="capitalizer" />
<to uri="direct:reverseAndTruncate"/>
</route>
<route>
<from uri="direct:reverseAndTruncate" />
<bean ref="reverser" />
<bean ref="truncater" />
<to uri="bean:endTimestamper?method=timestamp"/>
</route>
</camelContext>
<bean id="testDriver" class="camel.test.TestDriver">
<property name="producerTemplate" ref="myCamelTemplate" />
</bean>
</beans>
Claus Ibsen-2 wrote:
>
> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com>
> wrote:
>> Hi,
>>
>> If you don't want the TypeConverter to get invovled , you could use
>> MessageSupport.getBody() directly.
> Yes I am wondering if he has a .convertBodyTo in the route, so we need
> to see this.
>
> Or some other endpoint/producer trying to get the body as a special
> type, eg InputStream etc.
>
> But the route would really help.
>
>
>>
>> Willem
>>
>>
>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>>
>>>
>>> I'm not sure if it makes a difference but I'm not using JMS anywhere. In
>>> fact
>>> in this test everything is using "direct".
>>>
>>> Is there something I can do in the Spring DSL to hint to Camel that
>>> there
>>> is
>>> no conversion necessary?
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>> >
>>> > On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com>
>>> wrote:
>>> >>
>>> >> I've been seeing some performance problems with Camel 1.6.0 (I have
>>> not
>>> >> tried
>>> >> this with previous versions yet).
>>> >>
>>> >> My profiler is pointing the finger at MessageSupport.getBody,
>>> >> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>> >> specifically
>>> >>
>>> >> findTypeConverter is always throwing a
>>> >> NoTypeConversionAvailableException;
>>> >> which is then being caught and ignored in MessageSupport.getBody; at
>>> >> which
>>> >> point processing continues successfully.
>>> > This should be normal in situations where you ask the body to be a
>>> > specific type which cannot be converted to.
>>> > To help this can you show your route and what kind of JMS messages are
>>> > you sending.
>>> >
>>> > Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>>> > you can send whatever objects you like.
>>> > It has a rich type converter registry to be able to convert seamless
>>> > between types.
>>> >
>>> > This registry is loaded on demand, so you should make sure your start
>>> > profiling after Camel is "warm".
>>> >
>>> >
>>> >
>>> >>
>>> >> protected <T> T getBody(Class<T> type, Object body) is the specific
>>> >> getBody
>>> >> in question.
>>> >>
>>> >> Is this exception an expected behavior? It's weird how the catch
>>> block
>>> >> doesn't even log a warning. Should a converter have been found? My
>>> >> message
>>> >> payload is just a java.lang.String.
>>> > In the old days it returned null, but that did not work as the payload
>>> > you were trying to convert could be null, so it was a catch-22
>>> > situation.
>>> > So we added the exception.
>>> >
>>> > But if throwing exception is expensive we could maybe add a has test
>>> > to avoid this exception being thrown and caught in the MessageSupport.
>>> >
>>> > The exception is also meant for end users so they get a good exception
>>> > detailing the problem if they try to convert something into eg,
>>> > MyFooClass and its not possible to convert to it. Instead of returning
>>> > a null value as result.
>>> >
>>> >
>>> >>
>>> >> I suspect I've done something wrong but I don't know where to start
>>> >> looking.
>>> >> I'm concerned with this; as I'm comparing Camel to some other message
>>> >> routing solutions. This is making Camel take 40 times longer than the
>>> >> competition and I want to make sure I do a fair comparison.
>>> > We are currently rewamping the internal API in Camel 2.0 that leads us
>>> > up to a point where we can do performance improvements when Camel
>>> > routes exchanges.
>>> > Currently it does a bit of defensive copying when it moves message
>>> > from node to node. The revamped API lets us do some more clever stuff
>>> > there to improve the speed.
>>> >
>>> > So if you are testing, eg. JMS -> JMS and want it to be really fast
>>> > then of course pure JMS to JMS is faster than eg over Camel as its a
>>> > very flexible and transport/protocol agnostic framework. But
>>> > performance improvements is on our roadmap in 2.1.
>>> >
>>> >
>>> >> --
>>> >> View this message in context:
>>> >>
>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>> >> Sent from the Camel - Users mailing list archive at Nabble.com.
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Claus Ibsen
>>> > Apache Camel Committer
>>> >
>>> > Open Source Integration: http://fusesource.com
>>> > Blog: http://davsclaus.blogspot.com/
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
>
--
View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22308896.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <wi...@gmail.com> wrote:
> Hi,
>
> If you don't want the TypeConverter to get invovled , you could use
> MessageSupport.getBody() directly.
Yes I am wondering if he has a .convertBodyTo in the route, so we need
to see this.
Or some other endpoint/producer trying to get the body as a special
type, eg InputStream etc.
But the route would really help.
>
> Willem
>
>
> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>
>>
>> I'm not sure if it makes a difference but I'm not using JMS anywhere. In
>> fact
>> in this test everything is using "direct".
>>
>> Is there something I can do in the Spring DSL to hint to Camel that there
>> is
>> no conversion necessary?
>>
>>
>> Claus Ibsen-2 wrote:
>> >
>> > On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com> wrote:
>> >>
>> >> I've been seeing some performance problems with Camel 1.6.0 (I have not
>> >> tried
>> >> this with previous versions yet).
>> >>
>> >> My profiler is pointing the finger at MessageSupport.getBody,
>> >> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>> >> specifically
>> >>
>> >> findTypeConverter is always throwing a
>> >> NoTypeConversionAvailableException;
>> >> which is then being caught and ignored in MessageSupport.getBody; at
>> >> which
>> >> point processing continues successfully.
>> > This should be normal in situations where you ask the body to be a
>> > specific type which cannot be converted to.
>> > To help this can you show your route and what kind of JMS messages are
>> > you sending.
>> >
>> > Camel is payload type agnostics (eg dont have to be pure XML etc.) so
>> > you can send whatever objects you like.
>> > It has a rich type converter registry to be able to convert seamless
>> > between types.
>> >
>> > This registry is loaded on demand, so you should make sure your start
>> > profiling after Camel is "warm".
>> >
>> >
>> >
>> >>
>> >> protected <T> T getBody(Class<T> type, Object body) is the specific
>> >> getBody
>> >> in question.
>> >>
>> >> Is this exception an expected behavior? It's weird how the catch block
>> >> doesn't even log a warning. Should a converter have been found? My
>> >> message
>> >> payload is just a java.lang.String.
>> > In the old days it returned null, but that did not work as the payload
>> > you were trying to convert could be null, so it was a catch-22
>> > situation.
>> > So we added the exception.
>> >
>> > But if throwing exception is expensive we could maybe add a has test
>> > to avoid this exception being thrown and caught in the MessageSupport.
>> >
>> > The exception is also meant for end users so they get a good exception
>> > detailing the problem if they try to convert something into eg,
>> > MyFooClass and its not possible to convert to it. Instead of returning
>> > a null value as result.
>> >
>> >
>> >>
>> >> I suspect I've done something wrong but I don't know where to start
>> >> looking.
>> >> I'm concerned with this; as I'm comparing Camel to some other message
>> >> routing solutions. This is making Camel take 40 times longer than the
>> >> competition and I want to make sure I do a fair comparison.
>> > We are currently rewamping the internal API in Camel 2.0 that leads us
>> > up to a point where we can do performance improvements when Camel
>> > routes exchanges.
>> > Currently it does a bit of defensive copying when it moves message
>> > from node to node. The revamped API lets us do some more clever stuff
>> > there to improve the speed.
>> >
>> > So if you are testing, eg. JMS -> JMS and want it to be really fast
>> > then of course pure JMS to JMS is faster than eg over Camel as its a
>> > very flexible and transport/protocol agnostic framework. But
>> > performance improvements is on our roadmap in 2.1.
>> >
>> >
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>> >> Sent from the Camel - Users mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > Apache Camel Committer
>> >
>> > Open Source Integration: http://fusesource.com
>> > Blog: http://davsclaus.blogspot.com/
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
If you don't want the TypeConverter to get invovled , you could use
MessageSupport.getBody() directly.
Willem
On Tue, Mar 3, 2009 at 1:43 AM, paquettd <da...@lmco.com> wrote:
>
> I'm not sure if it makes a difference but I'm not using JMS anywhere. In
> fact
> in this test everything is using "direct".
>
> Is there something I can do in the Spring DSL to hint to Camel that there
> is
> no conversion necessary?
>
>
> Claus Ibsen-2 wrote:
> >
> > On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com> wrote:
> >>
> >> I've been seeing some performance problems with Camel 1.6.0 (I have not
> >> tried
> >> this with previous versions yet).
> >>
> >> My profiler is pointing the finger at MessageSupport.getBody,
> >> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
> >> specifically
> >>
> >> findTypeConverter is always throwing a
> >> NoTypeConversionAvailableException;
> >> which is then being caught and ignored in MessageSupport.getBody; at
> >> which
> >> point processing continues successfully.
> > This should be normal in situations where you ask the body to be a
> > specific type which cannot be converted to.
> > To help this can you show your route and what kind of JMS messages are
> > you sending.
> >
> > Camel is payload type agnostics (eg dont have to be pure XML etc.) so
> > you can send whatever objects you like.
> > It has a rich type converter registry to be able to convert seamless
> > between types.
> >
> > This registry is loaded on demand, so you should make sure your start
> > profiling after Camel is "warm".
> >
> >
> >
> >>
> >> protected <T> T getBody(Class<T> type, Object body) is the specific
> >> getBody
> >> in question.
> >>
> >> Is this exception an expected behavior? It's weird how the catch block
> >> doesn't even log a warning. Should a converter have been found? My
> >> message
> >> payload is just a java.lang.String.
> > In the old days it returned null, but that did not work as the payload
> > you were trying to convert could be null, so it was a catch-22
> > situation.
> > So we added the exception.
> >
> > But if throwing exception is expensive we could maybe add a has test
> > to avoid this exception being thrown and caught in the MessageSupport.
> >
> > The exception is also meant for end users so they get a good exception
> > detailing the problem if they try to convert something into eg,
> > MyFooClass and its not possible to convert to it. Instead of returning
> > a null value as result.
> >
> >
> >>
> >> I suspect I've done something wrong but I don't know where to start
> >> looking.
> >> I'm concerned with this; as I'm comparing Camel to some other message
> >> routing solutions. This is making Camel take 40 times longer than the
> >> competition and I want to make sure I do a fair comparison.
> > We are currently rewamping the internal API in Camel 2.0 that leads us
> > up to a point where we can do performance improvements when Camel
> > routes exchanges.
> > Currently it does a bit of defensive copying when it moves message
> > from node to node. The revamped API lets us do some more clever stuff
> > there to improve the speed.
> >
> > So if you are testing, eg. JMS -> JMS and want it to be really fast
> > then of course pure JMS to JMS is faster than eg over Camel as its a
> > very flexible and transport/protocol agnostic framework. But
> > performance improvements is on our roadmap in 2.1.
> >
> >
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
> >> Sent from the Camel - Users mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> >
> > --
> > Claus Ibsen
> > Apache Camel Committer
> >
> > Open Source Integration: http://fusesource.com
> > Blog: http://davsclaus.blogspot.com/
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by paquettd <da...@lmco.com>.
I'm not sure if it makes a difference but I'm not using JMS anywhere. In fact
in this test everything is using "direct".
Is there something I can do in the Spring DSL to hint to Camel that there is
no conversion necessary?
Claus Ibsen-2 wrote:
>
> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com> wrote:
>>
>> I've been seeing some performance problems with Camel 1.6.0 (I have not
>> tried
>> this with previous versions yet).
>>
>> My profiler is pointing the finger at MessageSupport.getBody,
>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>> specifically
>>
>> findTypeConverter is always throwing a
>> NoTypeConversionAvailableException;
>> which is then being caught and ignored in MessageSupport.getBody; at
>> which
>> point processing continues successfully.
> This should be normal in situations where you ask the body to be a
> specific type which cannot be converted to.
> To help this can you show your route and what kind of JMS messages are
> you sending.
>
> Camel is payload type agnostics (eg dont have to be pure XML etc.) so
> you can send whatever objects you like.
> It has a rich type converter registry to be able to convert seamless
> between types.
>
> This registry is loaded on demand, so you should make sure your start
> profiling after Camel is "warm".
>
>
>
>>
>> protected <T> T getBody(Class<T> type, Object body) is the specific
>> getBody
>> in question.
>>
>> Is this exception an expected behavior? It's weird how the catch block
>> doesn't even log a warning. Should a converter have been found? My
>> message
>> payload is just a java.lang.String.
> In the old days it returned null, but that did not work as the payload
> you were trying to convert could be null, so it was a catch-22
> situation.
> So we added the exception.
>
> But if throwing exception is expensive we could maybe add a has test
> to avoid this exception being thrown and caught in the MessageSupport.
>
> The exception is also meant for end users so they get a good exception
> detailing the problem if they try to convert something into eg,
> MyFooClass and its not possible to convert to it. Instead of returning
> a null value as result.
>
>
>>
>> I suspect I've done something wrong but I don't know where to start
>> looking.
>> I'm concerned with this; as I'm comparing Camel to some other message
>> routing solutions. This is making Camel take 40 times longer than the
>> competition and I want to make sure I do a fair comparison.
> We are currently rewamping the internal API in Camel 2.0 that leads us
> up to a point where we can do performance improvements when Camel
> routes exchanges.
> Currently it does a bit of defensive copying when it moves message
> from node to node. The revamped API lets us do some more clever stuff
> there to improve the speed.
>
> So if you are testing, eg. JMS -> JMS and want it to be really fast
> then of course pure JMS to JMS is faster than eg over Camel as its a
> very flexible and transport/protocol agnostic framework. But
> performance improvements is on our roadmap in 2.1.
>
>
>> --
>> View this message in context:
>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
>
--
View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance and MessageSupport.getBody (1.6.0)
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Mar 2, 2009 at 5:54 PM, paquettd <da...@lmco.com> wrote:
>
> I've been seeing some performance problems with Camel 1.6.0 (I have not tried
> this with previous versions yet).
>
> My profiler is pointing the finger at MessageSupport.getBody,
> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
> specifically
>
> findTypeConverter is always throwing a NoTypeConversionAvailableException;
> which is then being caught and ignored in MessageSupport.getBody; at which
> point processing continues successfully.
This should be normal in situations where you ask the body to be a
specific type which cannot be converted to.
To help this can you show your route and what kind of JMS messages are
you sending.
Camel is payload type agnostics (eg dont have to be pure XML etc.) so
you can send whatever objects you like.
It has a rich type converter registry to be able to convert seamless
between types.
This registry is loaded on demand, so you should make sure your start
profiling after Camel is "warm".
>
> protected <T> T getBody(Class<T> type, Object body) is the specific getBody
> in question.
>
> Is this exception an expected behavior? It's weird how the catch block
> doesn't even log a warning. Should a converter have been found? My message
> payload is just a java.lang.String.
In the old days it returned null, but that did not work as the payload
you were trying to convert could be null, so it was a catch-22
situation.
So we added the exception.
But if throwing exception is expensive we could maybe add a has test
to avoid this exception being thrown and caught in the MessageSupport.
The exception is also meant for end users so they get a good exception
detailing the problem if they try to convert something into eg,
MyFooClass and its not possible to convert to it. Instead of returning
a null value as result.
>
> I suspect I've done something wrong but I don't know where to start looking.
> I'm concerned with this; as I'm comparing Camel to some other message
> routing solutions. This is making Camel take 40 times longer than the
> competition and I want to make sure I do a fair comparison.
We are currently rewamping the internal API in Camel 2.0 that leads us
up to a point where we can do performance improvements when Camel
routes exchanges.
Currently it does a bit of defensive copying when it moves message
from node to node. The revamped API lets us do some more clever stuff
there to improve the speed.
So if you are testing, eg. JMS -> JMS and want it to be really fast
then of course pure JMS to JMS is faster than eg over Camel as its a
very flexible and transport/protocol agnostic framework. But
performance improvements is on our roadmap in 2.1.
> --
> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/