You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by rsahadevan <pr...@gmail.com> on 2017/05/31 16:08:27 UTC
ActiveMQ Embedded with Tomcat missing messages while restart
Hi All,
Thanks in Advance.
We have an ActiveMQ embedded running Tomcat with spring configuration. After
Tomcat server restarts few records from the persisted queue are getting
missed out. Below is my spring config. It seems the issue may be ActiveMQ
embedded starting faster than tomcat and is not getting Hibernate connection
for database. Could you please advise. While restart I am getting error like
this
java.lang.NullPointerException
at
org.codehaus.groovy.grails.orm.hibernate.transaction.PlatformTransactionManagerProxy.getTransaction(PlatformTransactionManagerProxy.java:22)
<bean id="persistenceAdapter"
class="org.apache.activemq.store.jdbc.JDBCPersistenceAdapter">
<property name="dataSource" ref="dataSource"/>
<property name="lockKeepAlivePeriod" value="2000"/>
<property name="createTablesOnStartup" value="false"/>
</bean>
<bean id="broker" class="org.apache.activemq.broker.BrokerService">
<property name="useJmx" value="false" />
<property name="persistent" value="true" />
<property name="persistenceAdapter" ref="persistenceAdapter"/>
<property name="transportConnectors">
<list>
<ref bean="tcpConnector" />
<ref bean="vmConnector" />
</list>
</property>
</bean>
<bean id="tcpConnector"
class="org.apache.activemq.broker.TransportConnector">
<property name="uri" value="tcp://localhost:61616"></property>
</bean>
<bean id="vmConnector"
class="org.apache.activemq.broker.TransportConnector">
<property name="uri" value="vm://localhost"></property>
</bean>
<bean id="jmsConnectionFactory"
class="org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy">
<property name="targetConnectionFactory">
<bean class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean
class="org.apache.activemq.ActiveMQConnectionFactory" depends-on="broker">
<property name="brokerURL" value="vm://localhost"/>
</bean>
</property>
</bean>
</property>
<property name="synchedLocalTransactionAllowed" value="true" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory">
<ref local="jmsConnectionFactory"/>
</property>
<property name="sessionTransacted" value="true" />
</bean>
Regards
Viraj
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by Tim Bain <tb...@alumni.duke.edu>.
Can you please post the full stack trace for that exception?
Also, is the dataSource bean defined in your webapp's Spring config file,
or are you getting it from Tomcat?
Tim
On May 31, 2017 10:32 AM, "rsahadevan" <pr...@gmail.com> wrote:
> Hi All,
>
> Thanks in Advance.
>
> We have an ActiveMQ embedded running Tomcat with spring configuration.
> After
> Tomcat server restarts few records from the persisted queue are getting
> missed out. Below is my spring config. It seems the issue may be ActiveMQ
> embedded starting faster than tomcat and is not getting Hibernate
> connection
> for database. Could you please advise. While restart I am getting error
> like
> this
> java.lang.NullPointerException
> at
> org.codehaus.groovy.grails.orm.hibernate.transaction.
> PlatformTransactionManagerProxy.getTransaction(
> PlatformTransactionManagerProxy.java:22)
>
> <bean id="persistenceAdapter"
> class="org.apache.activemq.store.jdbc.JDBCPersistenceAdapter">
> <property name="dataSource" ref="dataSource"/>
> <property name="lockKeepAlivePeriod" value="2000"/>
> <property name="createTablesOnStartup" value="false"/>
> </bean>
> <bean id="broker" class="org.apache.activemq.
> broker.BrokerService">
> <property name="useJmx" value="false" />
> <property name="persistent" value="true" />
> <property name="persistenceAdapter"
> ref="persistenceAdapter"/>
> <property name="transportConnectors">
> <list>
> <ref bean="tcpConnector" />
> <ref bean="vmConnector" />
> </list>
> </property>
> </bean>
>
> <bean id="tcpConnector"
> class="org.apache.activemq.broker.TransportConnector">
> <property name="uri" value="tcp://localhost:61616">
> </property>
> </bean>
> <bean id="vmConnector"
> class="org.apache.activemq.broker.TransportConnector">
> <property name="uri" value="vm://localhost"></property>
> </bean>
>
>
> <bean id="jmsConnectionFactory"
>
> class="org.springframework.jms.connection.TransactionAwareConnectionFact
> oryProxy">
> <property name="targetConnectionFactory">
> <bean class="org.apache.activemq.pool.PooledConnectionFactory"
> destroy-method="stop">
> <property name="connectionFactory">
> <bean
> class="org.apache.activemq.ActiveMQConnectionFactory" depends-on="broker">
> <property name="brokerURL" value="vm://localhost"/>
> </bean>
> </property>
> </bean>
> </property>
> <property name="synchedLocalTransactionAllowed" value="true" />
> </bean>
>
> <bean id="jmsTemplate" class="org.springframework.
> jms.core.JmsTemplate">
> <property name="connectionFactory">
> <ref local="jmsConnectionFactory"/>
> </property>
> <property name="sessionTransacted" value="true" />
> </bean>
>
> Regards
> Viraj
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-
> while-restart-tp4726828.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by Justin Bertram <jb...@apache.org>.
I don't see in your code where you're calling rollback either on any Hibernate or JMS resource.
Typically in this kind of scenario you'd want the consumption of the message and the database operation to be atomic to ensure the integrity of your data (i.e. one message equals one database operation). Without atomicity you risk losing or duplicating data. These kinds of guarantees are really easy to configure in a Java EE environment (e.g. using container-managed transactions and an XA JDBC datasource), but I'm not familiar enough with Spring to make any recommendations.
The bottom line here is that I think your application and/or architecture is the problem here and not the message broker.
Justin
----- Original Message -----
From: "rsahadevan" <pr...@gmail.com>
To: users@activemq.apache.org
Sent: Friday, June 2, 2017 2:20:07 PM
Subject: Re: ActiveMQ Embedded with Tomcat missing messages while restart
HI Justin,
What you understood is correct. Consumer is writing to database. I tried
with rollback and is not happening. Commit is failing in the DB and not
happening rollback into activemq_msgs. This is the reason I was thinking
about this approach . Please let me know anything I am missing in config
public void onMessage(Message message) {
try {
PlatformTransactionManager transactionManager =
Context.getBean(Context.Name.TRANSACTION_MANAGER);
TransactionStatus transaction =
transactionManager.getTransaction(new
DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRED));
updateDynamicBalance(new
Integer(message.getIntProperty("entityId")),new
Integer(message.getIntProperty("userId")),new
BigDecimal(message.getDoubleProperty("amount")),message.getStringProperty("mediationKey"));
transaction.flush();
return;
} catch (Exception e) {
LOG.error("Generating payment", e);
}
Spring Config
-----------------
<jms:listener-container connection-factory="jmsConnectionFactory"
transaction-manager="transactionManager" acknowledge="transacted">
<jms:listener ref="processBalanaceMDB"
destination="queue.jbilling.balance"/>
</jms:listener-container>
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726976.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by rsahadevan <pr...@gmail.com>.
HI Justin,
What you understood is correct. Consumer is writing to database. I tried
with rollback and is not happening. Commit is failing in the DB and not
happening rollback into activemq_msgs. This is the reason I was thinking
about this approach . Please let me know anything I am missing in config
public void onMessage(Message message) {
try {
PlatformTransactionManager transactionManager =
Context.getBean(Context.Name.TRANSACTION_MANAGER);
TransactionStatus transaction =
transactionManager.getTransaction(new
DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRED));
updateDynamicBalance(new
Integer(message.getIntProperty("entityId")),new
Integer(message.getIntProperty("userId")),new
BigDecimal(message.getDoubleProperty("amount")),message.getStringProperty("mediationKey"));
transaction.flush();
return;
} catch (Exception e) {
LOG.error("Generating payment", e);
}
Spring Config
-----------------
<jms:listener-container connection-factory="jmsConnectionFactory"
transaction-manager="transactionManager" acknowledge="transacted">
<jms:listener ref="processBalanaceMDB"
destination="queue.jbilling.balance"/>
</jms:listener-container>
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726976.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by Justin Bertram <jb...@apache.org>.
I don't understand the flow here. Is your JMS listener consuming messages and then attempting to insert them into a database using the invalid Hibernate session? If so, why don't you just rollback the session if the Hibernate operation fails and set a redelivery delay on the messages?
Justin
----- Original Message -----
From: "rsahadevan" <pr...@gmail.com>
To: users@activemq.apache.org
Sent: Friday, June 2, 2017 12:11:32 PM
Subject: Re: ActiveMQ Embedded with Tomcat missing messages while restart
HI All,
Identified the issue. JMS Listener fires before Hibernate is setup on Server
startup. Could you some one help on below points
1) How I can delay jms:listener-container for 3 minutes to start. This will
solve the issue. By the time hibernate session will be ready.
<jms:listener-container connection-factory="jmsConnectionFactory"
transaction-manager="transactionManager" acknowledge="transacted">
<jms:listener ref="processBalanaceMDB"
destination="queue.jbilling.balance"/>
<jms:listener ref="processPaymentMDB"
destination="queue.jbilling.processors"/>
<jms:listener ref="notificationMDB"
destination="queue.jbilling.notifications"/>
</jms:listener-container>
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726958.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by rsahadevan <pr...@gmail.com>.
HI All,
Identified the issue. JMS Listener fires before Hibernate is setup on Server
startup. Could you some one help on below points
1) How I can delay jms:listener-container for 3 minutes to start. This will
solve the issue. By the time hibernate session will be ready.
<jms:listener-container connection-factory="jmsConnectionFactory"
transaction-manager="transactionManager" acknowledge="transacted">
<jms:listener ref="processBalanaceMDB"
destination="queue.jbilling.balance"/>
<jms:listener ref="processPaymentMDB"
destination="queue.jbilling.processors"/>
<jms:listener ref="notificationMDB"
destination="queue.jbilling.notifications"/>
</jms:listener-container>
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726958.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by rsahadevan <pr...@gmail.com>.
Please find the error log also t3.gz
<http://activemq.2283324.n4.nabble.com/file/n4726860/t3.gz> . After
restarting tomcat embedded ActiveMQ is started immediately and sending
persisted messages to database. First few records are getting skipped
because of Nullpointer Exception.
Seems the Issue may be Hibernate connection not be available while
activeMq is started,
Please advise , What is the best solution to rollback these missed records
or delay activeMQ start till hibernate connection is available.
Regards
Rakesh
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726860.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Embedded with Tomcat missing messages while restart
Posted by rsahadevan <pr...@gmail.com>.
HI Tim,
Please find the attached spring resources.xml
<http://activemq.2283324.n4.nabble.com/file/n4726859/resources.xml>
resource.xml and grails datasource below
dataSource {
dialect = "org.hibernate.dialect.PostgreSQLDialect"
driverClassName = "org.postgresql.Driver"
username = dbUser
password = "postgres"
url = "jdbc:postgresql://${dbHost}:5432/${dbName}"
pooled = true
configClass = GrailsAnnotationConfiguration.class
}
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = true
cache.region.factory_class =
'net.sf.ehcache.hibernate.EhCacheRegionFactory'
}
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Embedded-with-Tomcat-missing-messages-while-restart-tp4726828p4726859.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.