You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by NickWK <ni...@misys.com> on 2010/06/23 11:29:24 UTC
Performance loss observed when using the ProducerTemplate to send
an exchange to an endpoint
I am using Apache Camel version 2.2.0-fuse-01-00 in the WebSphere Application
Server web container.
I am looking to use the Apache Camel ProducerTemplate class to send messages
to endpoints from within a Java class and have observed a significant
reduction in message throughput when using the
ProducerTemplate.send(Endpoint, Exchange) method over sending message to
endpoints directly in Java DSL. With a pure Java DSL solution I was
observing a throughput of 23 msgs/s and when writing to same Camel endpoints
from a JavaBean using a ProducerTemplate throughput fell to 4 msgs/s.
I have declared my ProducerTemplate as a bean in the Spring configuration
file as suggested in the documentation:
<camel-spring:template id="template"/>
In the JavaBean I lookup the template from Spring ApplicationContext ctx:
ProducerTemplate localProducer = (ProducerTemplate)ctx.getBean("template");
And write to a Camel endpoint using:
localProducer.send(endpoint, exchange);
With Spring handling the lifecycle of the ProducerTemplate.
Is this the correct way to use a ProducerTemplate?
Has anybody else experienced a performance hit when using the
ProducerTemplate?
Is their any alternative method available for writing to a camel producer
from a POJO?
--
View this message in context: http://camel.465427.n5.nabble.com/Performance-loss-observed-when-using-the-ProducerTemplate-to-send-an-exchange-to-an-endpoint-tp510843p510843.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance loss observed when using the ProducerTemplate to send
an exchange to an endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Nick
Did you have time to do any profiling or running some tests generating
logging output?
On Wed, Jun 23, 2010 at 4:43 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Maybe you could enable more logging and use that to see if you can
> spot an obvious time gap between the two?
>
> org.apache.camel.impl.ProducerCache=TRACE
> org.apache.camel.util.ServiceHelper=TRACE
>
> The ServiceHelper logs when producers/consumers etc. is being started stopped.
>
> You can also use an EventNotifier to log the time it takes to send a
> message to a endpoint.
> Also JMX gathers stats as well.
>
> Also a real profiler can help as well.
>
>
> On Wed, Jun 23, 2010 at 1:55 PM, NickWK <ni...@misys.com> wrote:
>>
>> Hi Claus,
>>
>> Thanks for your quick response.
>>
>> I using WebSphere MQ queues and a SQL Server 2005 database table as my
>> endpoints. I have made a code change this morning to cache the endpoints
>> once they have been created from the CamelContext using:
>>
>> Endpoint endpoint = camelContext.getEndpoint(uri)
>>
>> This along with a few other changes has improved the performance of the
>> routing from 4msgs/s to about 8msgs/sec but this is still well down on the
>> 23msgs/s I get with the pure Java DSL version. Please note that I am using
>> the WebSphere transaction manager to provide XA transaction coordination in
>> all my routes.
>>
>> My Camel route that uses the ProducerTemplate is given below:
>>
>> from("webspheremqxa:queue:input_queue").
>> transacted().
>> to("myParsingEngine").
>> to("myClient").
>> end();
>>
>> Where myParsingEngine processor loads the message in memory and myClient
>> processor manipulates the emssage and writes it to a WebSphere MQ queue and
>> a SQLServer table using the ProducerTemplate.
>>
>> My pure Java DSL route is given below:
>>
>> from("webspheremqxa:queue:input_queue3").
>> transacted().
>> to("myParsingEngine").
>> multicast().
>> pipeline(). // Route to the JDBC endpoint
>> choice().
>> when().method("myRoutingConditions","matches").
>> to("bean:myTransformEngine?method=doMappingOnly").
>> to("bean:mySelector?method=toSqlServerSql").
>> to("jdbc:mssql-2005-xa-ds").
>> //when().
>> //otherwise().
>> end().
>> end().
>> pipeline(). // Route to the JMS endpoint
>> choice().
>> when().method("myRoutingConditions","matches").
>> to("bean:myTransformEngine?method=doMappingOnly").
>> to("bean:myFormattingEngine?method=serialiseOnly").
>> to("webspheremqxa:queue:swift_output_queue2").
>> //when().
>> //otherwise().
>> end().
>> end().
>> end();
>>
>> Any further guidance would be greatly appreciated.
>>
>> Regards,
>>
>> Nick
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Performance-loss-observed-when-using-the-ProducerTemplate-to-send-an-exchange-to-an-endpoint-tp510843p510886.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Performance loss observed when using the ProducerTemplate to send
an exchange to an endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Maybe you could enable more logging and use that to see if you can
spot an obvious time gap between the two?
org.apache.camel.impl.ProducerCache=TRACE
org.apache.camel.util.ServiceHelper=TRACE
The ServiceHelper logs when producers/consumers etc. is being started stopped.
You can also use an EventNotifier to log the time it takes to send a
message to a endpoint.
Also JMX gathers stats as well.
Also a real profiler can help as well.
On Wed, Jun 23, 2010 at 1:55 PM, NickWK <ni...@misys.com> wrote:
>
> Hi Claus,
>
> Thanks for your quick response.
>
> I using WebSphere MQ queues and a SQL Server 2005 database table as my
> endpoints. I have made a code change this morning to cache the endpoints
> once they have been created from the CamelContext using:
>
> Endpoint endpoint = camelContext.getEndpoint(uri)
>
> This along with a few other changes has improved the performance of the
> routing from 4msgs/s to about 8msgs/sec but this is still well down on the
> 23msgs/s I get with the pure Java DSL version. Please note that I am using
> the WebSphere transaction manager to provide XA transaction coordination in
> all my routes.
>
> My Camel route that uses the ProducerTemplate is given below:
>
> from("webspheremqxa:queue:input_queue").
> transacted().
> to("myParsingEngine").
> to("myClient").
> end();
>
> Where myParsingEngine processor loads the message in memory and myClient
> processor manipulates the emssage and writes it to a WebSphere MQ queue and
> a SQLServer table using the ProducerTemplate.
>
> My pure Java DSL route is given below:
>
> from("webspheremqxa:queue:input_queue3").
> transacted().
> to("myParsingEngine").
> multicast().
> pipeline(). // Route to the JDBC endpoint
> choice().
> when().method("myRoutingConditions","matches").
> to("bean:myTransformEngine?method=doMappingOnly").
> to("bean:mySelector?method=toSqlServerSql").
> to("jdbc:mssql-2005-xa-ds").
> //when().
> //otherwise().
> end().
> end().
> pipeline(). // Route to the JMS endpoint
> choice().
> when().method("myRoutingConditions","matches").
> to("bean:myTransformEngine?method=doMappingOnly").
> to("bean:myFormattingEngine?method=serialiseOnly").
> to("webspheremqxa:queue:swift_output_queue2").
> //when().
> //otherwise().
> end().
> end().
> end();
>
> Any further guidance would be greatly appreciated.
>
> Regards,
>
> Nick
> --
> View this message in context: http://camel.465427.n5.nabble.com/Performance-loss-observed-when-using-the-ProducerTemplate-to-send-an-exchange-to-an-endpoint-tp510843p510886.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Performance loss observed when using the ProducerTemplate to
send an exchange to an endpoint
Posted by NickWK <ni...@misys.com>.
Hi Claus,
Thanks for your quick response.
I using WebSphere MQ queues and a SQL Server 2005 database table as my
endpoints. I have made a code change this morning to cache the endpoints
once they have been created from the CamelContext using:
Endpoint endpoint = camelContext.getEndpoint(uri)
This along with a few other changes has improved the performance of the
routing from 4msgs/s to about 8msgs/sec but this is still well down on the
23msgs/s I get with the pure Java DSL version. Please note that I am using
the WebSphere transaction manager to provide XA transaction coordination in
all my routes.
My Camel route that uses the ProducerTemplate is given below:
from("webspheremqxa:queue:input_queue").
transacted().
to("myParsingEngine").
to("myClient").
end();
Where myParsingEngine processor loads the message in memory and myClient
processor manipulates the emssage and writes it to a WebSphere MQ queue and
a SQLServer table using the ProducerTemplate.
My pure Java DSL route is given below:
from("webspheremqxa:queue:input_queue3").
transacted().
to("myParsingEngine").
multicast().
pipeline(). // Route to the JDBC endpoint
choice().
when().method("myRoutingConditions","matches").
to("bean:myTransformEngine?method=doMappingOnly").
to("bean:mySelector?method=toSqlServerSql").
to("jdbc:mssql-2005-xa-ds").
//when().
//otherwise().
end().
end().
pipeline(). // Route to the JMS endpoint
choice().
when().method("myRoutingConditions","matches").
to("bean:myTransformEngine?method=doMappingOnly").
to("bean:myFormattingEngine?method=serialiseOnly").
to("webspheremqxa:queue:swift_output_queue2").
//when().
//otherwise().
end().
end().
end();
Any further guidance would be greatly appreciated.
Regards,
Nick
--
View this message in context: http://camel.465427.n5.nabble.com/Performance-loss-observed-when-using-the-ProducerTemplate-to-send-an-exchange-to-an-endpoint-tp510843p510886.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Performance loss observed when using the ProducerTemplate to send
an exchange to an endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Can you show a bit more how you use the template and how you obtain
those endpoints.
And what kind of endpoints are those? Are you sending to a MQ or the likes?
Under the covers the Java DSL uses the same foundation as the template
so it ought to be able to run on par in speed.
On Wed, Jun 23, 2010 at 11:29 AM, NickWK <ni...@misys.com> wrote:
>
> I am using Apache Camel version 2.2.0-fuse-01-00 in the WebSphere Application
> Server web container.
>
> I am looking to use the Apache Camel ProducerTemplate class to send messages
> to endpoints from within a Java class and have observed a significant
> reduction in message throughput when using the
> ProducerTemplate.send(Endpoint, Exchange) method over sending message to
> endpoints directly in Java DSL. With a pure Java DSL solution I was
> observing a throughput of 23 msgs/s and when writing to same Camel endpoints
> from a JavaBean using a ProducerTemplate throughput fell to 4 msgs/s.
>
> I have declared my ProducerTemplate as a bean in the Spring configuration
> file as suggested in the documentation:
>
> <camel-spring:template id="template"/>
>
> In the JavaBean I lookup the template from Spring ApplicationContext ctx:
>
> ProducerTemplate localProducer = (ProducerTemplate)ctx.getBean("template");
>
> And write to a Camel endpoint using:
>
> localProducer.send(endpoint, exchange);
>
> With Spring handling the lifecycle of the ProducerTemplate.
>
> Is this the correct way to use a ProducerTemplate?
> Has anybody else experienced a performance hit when using the
> ProducerTemplate?
> Is their any alternative method available for writing to a camel producer
> from a POJO?
> --
> View this message in context: http://camel.465427.n5.nabble.com/Performance-loss-observed-when-using-the-ProducerTemplate-to-send-an-exchange-to-an-endpoint-tp510843p510843.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus