You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by dancsi <an...@epam.com> on 2014/05/30 17:18:34 UTC
Best Efforts 1 PC pattern - from MQ to DB
I'm trying to implement/configure Best Efforts 1PC pattern for
"transactional" consuming of MQ (IBM/WepSphere) messages and
inserting/updating the DB (Oracle).
Shall I use
org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy
over the MQ connection factory with Spring data source tx manager to handle
the DB-side of the transaction?
Does transacted route configuration apply in this case?
Thanks,
Dan
--
View this message in context: http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by Jeff Bischoff <jb...@wdtablesystems.com>.
I'm trying to figure out the same thing (although I'm working with AMQ,
not IBM). Queues seem pretty straightforward, but the topics seem very
sensitive to the caching/transaction settings. Don't know the right
answer, but interested to know.
JB
On 6/4/14 10:24 AM, "dancsi" <an...@epam.com> wrote:
>There is just one more thing I don't see clearly. I found quite
>contradicting
>information about the use of pooled/cached connection factories.
>
>First, at
>http://tmielke.blogspot.com/2012/03/camel-jms-with-transactions-lessons.ht
>ml
>I found that it's recommended to use pooled connection factory with Camel
>JMS and transactions.
>"When using a camel-jms configuration as above, its really important to
>use
>a pooled ConnectionFactory such as ActiveMQs PooledConnectionFactory or
>Springs CachingConnectionFactory. Not using a pooled ConnectionFactory
>means
>you will open a new JMS connection and session to the broker for every
>transaction / message to be received."
>
>From this, it seems to be obvious that I must use CachingConnectionFactory
>(as the broker I use is IBM/WebSphere MQ, I cannot use the
>PooledConnectionFactory).
>
>On the other hand, another source states that it isn't a good idea to use
>CachingConnectionFactory for message listeners/consumers.
>"It is generally not a good idea to use a caching connection factory with
>a
>listener container, especially when using maxConcurrentConsumers >
>concurrentConsumers - you can end up with cached consumers in the cache,
>which get messages where there is no listener, and such messages can get
>"stuck".
>So, don't use a CCF in this case, it's really intended for use on the
>producer side."
>http://stackoverflow.com/questions/18863057/closing-session-when-using-spr
>ings-cachingconnectionfactory
>
>Besides, another forum post recommends using WMQ's internal pooling
>instead.
>http://camel.465427.n5.nabble.com/New-JMS-connection-being-created-for-eve
>ry-message-td5637735.html
>
>So I'm rather mixed whether I should or shouldn't use
>CachingConnectionFactory. Can you give any direction?
>
>My configuration so far is as follows:
><endpoint id="inboundMQMessages" uri="inboundMQ:queue:${queueName}"/>
>
><bean:bean id="inboundMQ"
>class="org.apache.camel.component.jms.JmsComponent">
><bean:property name="connectionFactory" ref="mqConnectionFactory"/>
><bean:property name="transacted" value="true"/>
><bean:property name="transactionManager" ref="jmsTxManager"/>
><bean:property name="cacheLevelName" value="CACHE_CONSUMER"/>
><bean:property name="concurrentConsumers" value="10"/>
><bean:property name="maxConcurrentConsumers" value="20"/>
></bean:bean>
>
><bean:bean id="requiredJmsTx"
>class="org.apache.camel.spring.spi.SpringTransactionPolicy">
><bean:property name="transactionManager" ref="jmsTxManager"/>
><bean:property name="propagationBehaviorName"
>value="PROPAGATION_REQUIRED"/>
></bean:bean>
>
><bean:bean id="jmsTxManager"
>class="org.springframework.jms.connection.JmsTransactionManager">
> <bean:property name="connectionFactory" ref="mqConnectionFactory"/>
></bean:bean>
>
><bean:bean id="mqConnectionFactory"
>class="com.ibm.mq.jms.MQQueueConnectionFactory">
><bean:property name="transportType">
><util:constant static-field="${mqTransportType}" />
></bean:property>
><bean:property name="hostName" value="${mqHostName}" />
><bean:property name="port" value="${mqPort}" />
><bean:property name="queueManager" value="${mqQueueManager}" />
><bean:property name="channel" value="${mqSubscriberChannel}" />
><bean:property name="useConnectionPooling" value="true" />
><bean:property name="pollingInterval" value="${mqPollingInterval}" />
><bean:property name="rescanInterval" value="${mqRescanInterval}" />
></bean:bean>
>
>
>
>
>
>--
>View this message in context:
>http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-
>tp5751757p5751879.html
>Sent from the Camel - Users mailing list archive at Nabble.com.
________________________________
Please note: This email, including attachments, contains information which may be confidential or legally privileged and is only for the use of the individual or entity to whom it is properly addressed. Any unauthorized review, use, disclosure, copying, or distribution is prohibited. If you have reason to believe that you have received this communication in error, or that it may be misaddressed or not intended for you, please destroy it and notify the sender immediately. Thank you.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by David Karlsen <da...@gmail.com>.
We use only the IBM one and it works fine
6. Juni 2014 16:15 skrev "dancsi" <an...@epam.com> følgende:
> Anyone having experience with IBM/WebSphere MQ? Shall I rely on WMQ
> Connection Factory caching or shall I rather configure Spring's
> CachingConnectionFactory?
>
> Thanks a lot,
> Dan
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757p5751997.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by Jeff Bischoff <jb...@wdtablesystems.com>.
Someone once told me to avoid using Spring's connection factories with
Camel. Not sure of the why though.
JB
On 6/6/14 10:14 AM, "dancsi" <an...@epam.com> wrote:
>Anyone having experience with IBM/WebSphere MQ? Shall I rely on WMQ
>Connection Factory caching or shall I rather configure Spring's
>CachingConnectionFactory?
>
>Thanks a lot,
>Dan
>
>
>
>--
>View this message in context:
>http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-
>tp5751757p5751997.html
>Sent from the Camel - Users mailing list archive at Nabble.com.
________________________________
Please take note: This email, including attachments, contains information which may be confidential or legally privileged and is only for the use of the individual or entity to whom it is properly addressed. Any unauthorized review, use, disclosure, copying, or distribution is prohibited. If you have reason to believe that you have received this communication in error, or that it may be misaddressed or not intended for you, please destroy it and notify the sender immediately. Thank you.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by dancsi <an...@epam.com>.
Anyone having experience with IBM/WebSphere MQ? Shall I rely on WMQ
Connection Factory caching or shall I rather configure Spring's
CachingConnectionFactory?
Thanks a lot,
Dan
--
View this message in context: http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757p5751997.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by dancsi <an...@epam.com>.
There is just one more thing I don't see clearly. I found quite contradicting
information about the use of pooled/cached connection factories.
First, at
http://tmielke.blogspot.com/2012/03/camel-jms-with-transactions-lessons.html
I found that it's recommended to use pooled connection factory with Camel
JMS and transactions.
"When using a camel-jms configuration as above, its really important to use
a pooled ConnectionFactory such as ActiveMQs PooledConnectionFactory or
Springs CachingConnectionFactory. Not using a pooled ConnectionFactory means
you will open a new JMS connection and session to the broker for every
transaction / message to be received."
>From this, it seems to be obvious that I must use CachingConnectionFactory
(as the broker I use is IBM/WebSphere MQ, I cannot use the
PooledConnectionFactory).
On the other hand, another source states that it isn't a good idea to use
CachingConnectionFactory for message listeners/consumers.
"It is generally not a good idea to use a caching connection factory with a
listener container, especially when using maxConcurrentConsumers >
concurrentConsumers - you can end up with cached consumers in the cache,
which get messages where there is no listener, and such messages can get
"stuck".
So, don't use a CCF in this case, it's really intended for use on the
producer side."
http://stackoverflow.com/questions/18863057/closing-session-when-using-springs-cachingconnectionfactory
Besides, another forum post recommends using WMQ's internal pooling instead.
http://camel.465427.n5.nabble.com/New-JMS-connection-being-created-for-every-message-td5637735.html
So I'm rather mixed whether I should or shouldn't use
CachingConnectionFactory. Can you give any direction?
My configuration so far is as follows:
<endpoint id="inboundMQMessages" uri="inboundMQ:queue:${queueName}"/>
<bean:bean id="inboundMQ"
class="org.apache.camel.component.jms.JmsComponent">
<bean:property name="connectionFactory" ref="mqConnectionFactory"/>
<bean:property name="transacted" value="true"/>
<bean:property name="transactionManager" ref="jmsTxManager"/>
<bean:property name="cacheLevelName" value="CACHE_CONSUMER"/>
<bean:property name="concurrentConsumers" value="10"/>
<bean:property name="maxConcurrentConsumers" value="20"/>
</bean:bean>
<bean:bean id="requiredJmsTx"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<bean:property name="transactionManager" ref="jmsTxManager"/>
<bean:property name="propagationBehaviorName"
value="PROPAGATION_REQUIRED"/>
</bean:bean>
<bean:bean id="jmsTxManager"
class="org.springframework.jms.connection.JmsTransactionManager">
<bean:property name="connectionFactory" ref="mqConnectionFactory"/>
</bean:bean>
<bean:bean id="mqConnectionFactory"
class="com.ibm.mq.jms.MQQueueConnectionFactory">
<bean:property name="transportType">
<util:constant static-field="${mqTransportType}" />
</bean:property>
<bean:property name="hostName" value="${mqHostName}" />
<bean:property name="port" value="${mqPort}" />
<bean:property name="queueManager" value="${mqQueueManager}" />
<bean:property name="channel" value="${mqSubscriberChannel}" />
<bean:property name="useConnectionPooling" value="true" />
<bean:property name="pollingInterval" value="${mqPollingInterval}" />
<bean:property name="rescanInterval" value="${mqRescanInterval}" />
</bean:bean>
--
View this message in context: http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757p5751879.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by dancsi <an...@epam.com>.
Thanks a lot for your help. That's exactly what I needed to solve the
problem.
Dan
--
View this message in context: http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757p5751841.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Best Efforts 1 PC pattern - from MQ to DB
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
If the insert/update into the DB is only one write operation and its
the last in the route, then you can just use the JMS transactions. As
if write to DB fails it will thrown an exception and cause the JMS to
rollback.
If write success then no exception and the JMS can ack the message.
If you have a copy of Camel in Action, then chapter 9 covers all about
transaction and also this 1PC use-case, you talk about.
And the Camel docs has a bit about TX too at
http://camel.apache.org/transactional-client.html
On Fri, May 30, 2014 at 5:18 PM, dancsi <an...@epam.com> wrote:
> I'm trying to implement/configure Best Efforts 1PC pattern for
> "transactional" consuming of MQ (IBM/WepSphere) messages and
> inserting/updating the DB (Oracle).
>
> Shall I use
> org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy
> over the MQ connection factory with Spring data source tx manager to handle
> the DB-side of the transaction?
> Does transacted route configuration apply in this case?
>
> Thanks,
> Dan
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Best-Efforts-1-PC-pattern-from-MQ-to-DB-tp5751757.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/