You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by deepak_a <an...@gmail.com> on 2015/11/12 12:24:15 UTC
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
All,
I read the below link - and it appears this lets me batch messages during
consumption or publication
http://camel.apache.org/sjms.html
i.e. (multiple messages in one transaction).
But unfortunately this does not support XA (I am using Atomikos as
Transaction Manager)
https://issues.apache.org/jira/browse/CAMEL-6108
There is an open ticket for this – so likely to be available in future…..but
no ETA
So long story short – out-of-the box we do not have support for grouping
messages in one transaction.
Has any one found a work around for this? Would using spring-batch be
beneficial?
regards
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773715.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Antonin Stefanutti <an...@stefanutti.fr>.
It should be in spring-batch-integration.
> On 18 Nov 2015, at 18:52, deepak_a <an...@gmail.com> wrote:
>
> Hi,
>
> I have made a thorough search but unable to find
> docs.spring.io/spring-batch/apidocs/org/springframework/batch/container/jms/BatchMessageListenerContainer.html
>
> in the spring-batch/spring-batch-core library. (although the javadocs says
> its available)
>
> I don't see the package org/springframework/batch/container at all in the
> library.
> I am using version 3.0.5
>
>
> regards
> D
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774072.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by prajath <pr...@gmail.com>.
Hi ,
I tried to use BatchMessageListenerContainer with camel still it is picking
up single message from queue per transaction, could you please let me know
what I am doing wrong here.
Here is my code snippet
Look like BatchMessageListenerContainer not deployed in Maven central.
I have copied source code from spring-batch-infrastructure-tests module
And setup TransactionInterceptor and RepeatOperationsInterceptor as per
there documentation
<bean id="transactionInterceptor" class="org....TransactionInterceptor">
<property name="transactionManager" ref="jtaTransactionManager" />
<property name="transactionAttributes" value="*=PROPAGATION_REQUIRED" />
</bean>
<bean id="repeatOperationsInterceptor"
class="org.....RepeatOperationsInterceptor">
<property name="repeatOperations" ref="batchTemplate" />
</bean>
<bean id="batchTemplate" class="org.....RepeatTemplate">
<property name="completionPolicy">
<bean
class="org.springframework.batch.repeat.policy.SimpleCompletionPolicy">
<constructor-arg value="100" />
</bean>
</property>
</bean>
I was expecting 100 message per transaction .....
----BatchListenerContainerFactory---
public AbstractMessageListenerContainer createMessageListenerContainer(
JmsEndpoint arg0) {
BatchMessageListenerContainer batchMessageListenerContainer=new
BatchMessageListenerContainer();
batchMessageListenerContainer.setTransactionManager(transactionManager);
batchMessageListenerContainer.setConnectionFactory(connectionFactory);
batchMessageListenerContainer.setMessageListener(messageListener);
batchMessageListenerContainer.setDestinationName(destinationName);
batchMessageListenerContainer.setConcurrentConsumers(1);
batchMessageListenerContainer.setReceiveTimeout(3000);
batchMessageListenerContainer.setSessionTransacted(true);
Advice[] advices = new Advice[2];
advices[0]=transactionInterceptor;
advices[1]=repeatOperationsInterceptor;
batchMessageListenerContainer.setAdviceChain(advices);
return batchMessageListenerContainer;
}
------Camel root----
<camel:from
uri="jms:queue:queue.incomingEvents?messageListenerContainerFactoryRef=#batchListenerContainerFactory&transacted=true"
/>
<transacted ref="PROPAGATION_REQUIRES_NEW" />
<camel:to uri="stream:out"/>
regards,
pp
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774390.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Hi,
I have made a thorough search but unable to find
docs.spring.io/spring-batch/apidocs/org/springframework/batch/container/jms/BatchMessageListenerContainer.html
in the spring-batch/spring-batch-core library. (although the javadocs says
its available)
I don't see the package org/springframework/batch/container at all in the
library.
I am using version 3.0.5
regards
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774072.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Antonin Stefanutti <an...@stefanutti.fr>.
I don't know of any public / shareable example of setting a custom MessageListenerContainer unfortunately.
That being said, from the Camel route standpoint, you should write something like:
from://jms?consumerType=Custom&messageListenerContainerFactoryRef=…
This is documented in http://camel.apache.org/jms.html#JMS-Alltheotheroptions.
> On 18 Nov 2015, at 16:43, deepak_a <an...@gmail.com> wrote:
>
> Hi,
>
> Thanks for the reply, the XA connection factory is in place,
> Can you pls point to an example on how to override the default
> messageListener?
>
> (I have checked in forum - I am not able to see anyone done this in the
> past)
>
> regards
> D
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774063.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Hi,
Thanks for the reply, the XA connection factory is in place,
Can you pls point to an example on how to override the default
messageListener?
(I have checked in forum - I am not able to see anyone done this in the
past)
regards
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774063.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Antonin Stefanutti <an...@stefanutti.fr>.
You could use BatchMessageListenerContainer [1] instead of DefaultMessageListenerContainer.
An option to provide a specific MessageListenerContainer is to used the 'consumerType' parameter from the JMS component and set it to 'Custom' and provide the option 'messageListenerContainerFactoryRef' [2].
[1] http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/container/jms/BatchMessageListenerContainer.html
[2] http://camel.apache.org/jms.html#JMS-Alltheotheroptions
> On 18 Nov 2015, at 14:17, deepak_a <an...@gmail.com> wrote:
>
> Hi,
>
> As mentioned in my previous post - I am developing a bridge that will set
> b/w ActiveMQ and TIBCO EMS.
>
> My aim is to use a XATransactionManager (like Atomikos) that will be common
> for the JMS Session that will consume from ActiveMQ and the JMS Session that
> will be publish to TIBCO EMS.
>
> With the intermediate Queue - I still cannot achieve the above (especially
> when I want to group n messages into one transaction - since I want to
> achieve high throughput)
>
> Is there any other work-around any one can recommend?
>
> regards
> D
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774044.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Hi,
As mentioned in my previous post - I am developing a bridge that will set
b/w ActiveMQ and TIBCO EMS.
My aim is to use a XATransactionManager (like Atomikos) that will be common
for the JMS Session that will consume from ActiveMQ and the JMS Session that
will be publish to TIBCO EMS.
With the intermediate Queue - I still cannot achieve the above (especially
when I want to group n messages into one transaction - since I want to
achieve high throughput)
Is there any other work-around any one can recommend?
regards
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5774044.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Thanks, but I would have to compromise on performance right?
Because of the additional hop / intermediate queue?
regards,
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773916.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Claus Ibsen <cl...@gmail.com>.
Use batch-sjms with a local TX to do N messages into a 1 message and
send that to a queue, in a TX which it supports today.
Then from the new queue you can consume with XA, eg use an
intermediate queue in between.
On Mon, Nov 16, 2015 at 10:34 AM, deepak_a <an...@gmail.com> wrote:
> Hi Jakub,
>
> Thanks for the reply, that's unfortunate because XA is a must for me, and to
> improve performance I need to group number of messages in a Transaction.
>
> Is there a work around some one can suggest - would using Spring-batch help
> achieve this? Any suggestions will be helpful.
>
>
> regards
> D
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773913.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Hi Jakub,
Thanks for the reply, that's unfortunate because XA is a must for me, and to
improve performance I need to group number of messages in a Transaction.
Is there a work around some one can suggest - would using Spring-batch help
achieve this? Any suggestions will be helpful.
regards
D
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773913.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Jakub Korab <ja...@gmail.com>.
Hi Deepak,
The SJMS Batch component is included in the same library, but it's a
different Component implementation specifically for consuming batches of
messages. It works using local JMS transactions only, and does not
integrate with XA.
Jakub
On 12/11/15 11:59, deepak_a wrote:
> Thanks Claus,
>
> The dependency looks exactly same as SJMS, does this mean SJMS has been
> enhanced/re-branded as SJMS-batch? in version 2.16?
>
>
>
>
> <dependency>
> <groupId>org.apache.camel</groupId>
> <artifactId>camel-sjms</artifactId>
> <version>x.x.x</version>
>
> </dependency>
>
> Also can you pls point me to the Javadoc? I can only see Javadoc available
> for 2.15.
> I need to check if there is support for XA transactions....
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773717.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by deepak_a <an...@gmail.com>.
Thanks Claus,
The dependency looks exactly same as SJMS, does this mean SJMS has been
enhanced/re-branded as SJMS-batch? in version 2.16?
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-sjms</artifactId>
<version>x.x.x</version>
</dependency>
Also can you pls point me to the Javadoc? I can only see Javadoc available
for 2.15.
I need to check if there is support for XA transactions....
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773717.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to improve throughput by grouping messages in a Transaction
(in JMS)
Posted by Claus Ibsen <cl...@gmail.com>.
See
http://camel.apache.org/sjms-batch.html
On Thu, Nov 12, 2015 at 12:24 PM, deepak_a <an...@gmail.com> wrote:
> All,
>
> I read the below link - and it appears this lets me batch messages during
> consumption or publication
> http://camel.apache.org/sjms.html
> i.e. (multiple messages in one transaction).
>
> But unfortunately this does not support XA (I am using Atomikos as
> Transaction Manager)
>
> https://issues.apache.org/jira/browse/CAMEL-6108
>
> There is an open ticket for this – so likely to be available in future…..but
> no ETA
> So long story short – out-of-the box we do not have support for grouping
> messages in one transaction.
>
> Has any one found a work around for this? Would using spring-batch be
> beneficial?
>
>
> regards
> D
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773715.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2