You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Leonardo K. Shikida" <sh...@gmail.com> on 2015/09/11 19:33:59 UTC
need advice for single queue configuration
Hi
I need a simple queue.
This queue will be persistent and it will use a database in the backend to
store the messages.
It will receive produced messages small in size (just one or two string
attributes) in bursts of 10~100.
It will consume messages by a pool of 100~300 message driven beans (tomee).
Each message will be consumed and will trigger a long-running job that may
take from 1 minute to 1 hour to complete. If the job fails, I want the
message to return to the queue so it can be consumed again, no matter how
many retries.
I don't want to send invalid messages to a "poisoned messages queue".
Architecture is simple. The queue will run in a single activeMQ instance
and consumers are MDBs from a single TomEE+ web application.
So, here's what I have so far
activemq.xml=========================================
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true">
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy
limit="1000" />
</pendingMessageLimitStrategy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true"
memoryLimit="1mb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#oracle-ds" />
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="1 gb" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb" />
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb" />
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="tcp" uri="tcp://0.0.0.0:61616" />
</transportConnectors>
</broker>
<bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:@localhost:1521:XE" />
<property name="username" value="*****" />
<property name="password" value="******" />
<property name="poolPreparedStatements" value="true" />
<property name="maxActive" value="100" />
</bean>
tomee.xml=========================================
<Resource id="Default JMS Resource Adapter"
type="ActiveMQResourceAdapter">
BrokerXmlConfig = broker:(tcp://localhost:61616)
ServerUrl = tcp://localhost:61616
ThreadPoolSize = 200
</Resource>
<Resource id="MyJmsConnectionFactory"
type="javax.jms.ConnectionFactory">
ResourceAdapter = Default\ JMS\ Resource\ Adapter
PoolMaxSize = 200
</Resource>
<Resource id="JobQueue" type="javax.jms.Queue" />
MDB=========================================
@MessageDriven(activationConfig = {
@ActivationConfigProperty(propertyName = "destinationType",
propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "destination",
propertyValue = "JobQueue"),
@ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
propertyValue = "1"),
@ActivationConfigProperty(propertyName = "maxSessions",
propertyValue = "200"),
@ActivationConfigProperty(propertyName = "acknowledgeMode",
propertyValue = "Auto-acknowledge") })
=========================================
With this configuration, I am still facing some interruptions.
Sometimes, TomEE simply stops consuming the queue (once or twice per week),
so I have to restart it. I am still checking the logs for anything
suspicious.
I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone AMQ
5.10 stable in a linux environment and oracle JDK 7.
But I'd like some advice on how to properly set up activemq / tomee in this
scenario (that's why I am sending this email to both activemq and tomee
lists)
Thank you very much, any help is welcome.
[]
Leo
Re: need advice for single queue configuration
Posted by Tim Bain <tb...@alumni.duke.edu>.
If you want redelivery to other consumers rather than just the one to which
the message was first delivered (which is how I interpreted your paragraph
about message-driven beans) and you can live with out-of-order delivery,
I'd think you'd want to configure broker redelivery as described at the
bottom of
http://activemq.apache.org/message-redelivery-and-dlq-handling.html.
When TomEE "simply stops consuming the queue", have you taken a thread dump
to see what its threads are doing? Is it possible that every MDB is in the
middle of processing a 1-hour message?
Have you used a JMX viewer such as JConsole to examine the subscriptions on
your destination on the broker when TomEE is having problems, to see if
it's still subscribed?
Tim
On Sep 11, 2015 11:34 AM, "Leonardo K. Shikida" <sh...@gmail.com> wrote:
> Hi
>
> I need a simple queue.
>
> This queue will be persistent and it will use a database in the backend to
> store the messages.
>
> It will receive produced messages small in size (just one or two string
> attributes) in bursts of 10~100.
>
> It will consume messages by a pool of 100~300 message driven beans (tomee).
> Each message will be consumed and will trigger a long-running job that may
> take from 1 minute to 1 hour to complete. If the job fails, I want the
> message to return to the queue so it can be consumed again, no matter how
> many retries.
>
> I don't want to send invalid messages to a "poisoned messages queue".
>
> Architecture is simple. The queue will run in a single activeMQ instance
> and consumers are MDBs from a single TomEE+ web application.
>
> So, here's what I have so far
>
> activemq.xml=========================================
>
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.data}">
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true">
> <pendingMessageLimitStrategy>
> <constantPendingMessageLimitStrategy
> limit="1000" />
> </pendingMessageLimitStrategy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb">
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <persistenceAdapter>
> <jdbcPersistenceAdapter dataSource="#oracle-ds" />
> </persistenceAdapter>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="1 gb" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="100 gb" />
> </storeUsage>
> <tempUsage>
> <tempUsage limit="50 gb" />
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="tcp" uri="tcp://0.0.0.0:61616" />
> </transportConnectors>
> </broker>
>
> <bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource"
> destroy-method="close">
> <property name="driverClassName" value="oracle.jdbc.OracleDriver"
> />
> <property name="url" value="jdbc:oracle:thin:@localhost:1521:XE"
> />
> <property name="username" value="*****" />
> <property name="password" value="******" />
> <property name="poolPreparedStatements" value="true" />
> <property name="maxActive" value="100" />
> </bean>
>
> tomee.xml=========================================
>
> <Resource id="Default JMS Resource Adapter"
> type="ActiveMQResourceAdapter">
> BrokerXmlConfig = broker:(tcp://localhost:61616)
> ServerUrl = tcp://localhost:61616
> ThreadPoolSize = 200
> </Resource>
>
> <Resource id="MyJmsConnectionFactory"
> type="javax.jms.ConnectionFactory">
> ResourceAdapter = Default\ JMS\ Resource\ Adapter
> PoolMaxSize = 200
> </Resource>
>
> <Resource id="JobQueue" type="javax.jms.Queue" />
>
> MDB=========================================
>
> @MessageDriven(activationConfig = {
> @ActivationConfigProperty(propertyName = "destinationType",
> propertyValue = "javax.jms.Queue"),
> @ActivationConfigProperty(propertyName = "destination",
> propertyValue = "JobQueue"),
> @ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
> propertyValue = "1"),
> @ActivationConfigProperty(propertyName = "maxSessions",
> propertyValue = "200"),
> @ActivationConfigProperty(propertyName = "acknowledgeMode",
> propertyValue = "Auto-acknowledge") })
>
> =========================================
>
> With this configuration, I am still facing some interruptions.
>
> Sometimes, TomEE simply stops consuming the queue (once or twice per week),
> so I have to restart it. I am still checking the logs for anything
> suspicious.
>
> I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone AMQ
> 5.10 stable in a linux environment and oracle JDK 7.
>
> But I'd like some advice on how to properly set up activemq / tomee in this
> scenario (that's why I am sending this email to both activemq and tomee
> lists)
>
> Thank you very much, any help is welcome.
>
> []
>
> Leo
>
Re: need advice for single queue configuration
Posted by Tim Bain <tb...@alumni.duke.edu>.
If you want redelivery to other consumers rather than just the one to which
the message was first delivered (which is how I interpreted your paragraph
about message-driven beans) and you can live with out-of-order delivery,
I'd think you'd want to configure broker redelivery as described at the
bottom of
http://activemq.apache.org/message-redelivery-and-dlq-handling.html.
When TomEE "simply stops consuming the queue", have you taken a thread dump
to see what its threads are doing? Is it possible that every MDB is in the
middle of processing a 1-hour message?
Have you used a JMX viewer such as JConsole to examine the subscriptions on
your destination on the broker when TomEE is having problems, to see if
it's still subscribed?
Tim
On Sep 11, 2015 11:34 AM, "Leonardo K. Shikida" <sh...@gmail.com> wrote:
> Hi
>
> I need a simple queue.
>
> This queue will be persistent and it will use a database in the backend to
> store the messages.
>
> It will receive produced messages small in size (just one or two string
> attributes) in bursts of 10~100.
>
> It will consume messages by a pool of 100~300 message driven beans (tomee).
> Each message will be consumed and will trigger a long-running job that may
> take from 1 minute to 1 hour to complete. If the job fails, I want the
> message to return to the queue so it can be consumed again, no matter how
> many retries.
>
> I don't want to send invalid messages to a "poisoned messages queue".
>
> Architecture is simple. The queue will run in a single activeMQ instance
> and consumers are MDBs from a single TomEE+ web application.
>
> So, here's what I have so far
>
> activemq.xml=========================================
>
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.data}">
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true">
> <pendingMessageLimitStrategy>
> <constantPendingMessageLimitStrategy
> limit="1000" />
> </pendingMessageLimitStrategy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb">
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <persistenceAdapter>
> <jdbcPersistenceAdapter dataSource="#oracle-ds" />
> </persistenceAdapter>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="1 gb" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="100 gb" />
> </storeUsage>
> <tempUsage>
> <tempUsage limit="50 gb" />
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="tcp" uri="tcp://0.0.0.0:61616" />
> </transportConnectors>
> </broker>
>
> <bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource"
> destroy-method="close">
> <property name="driverClassName" value="oracle.jdbc.OracleDriver"
> />
> <property name="url" value="jdbc:oracle:thin:@localhost:1521:XE"
> />
> <property name="username" value="*****" />
> <property name="password" value="******" />
> <property name="poolPreparedStatements" value="true" />
> <property name="maxActive" value="100" />
> </bean>
>
> tomee.xml=========================================
>
> <Resource id="Default JMS Resource Adapter"
> type="ActiveMQResourceAdapter">
> BrokerXmlConfig = broker:(tcp://localhost:61616)
> ServerUrl = tcp://localhost:61616
> ThreadPoolSize = 200
> </Resource>
>
> <Resource id="MyJmsConnectionFactory"
> type="javax.jms.ConnectionFactory">
> ResourceAdapter = Default\ JMS\ Resource\ Adapter
> PoolMaxSize = 200
> </Resource>
>
> <Resource id="JobQueue" type="javax.jms.Queue" />
>
> MDB=========================================
>
> @MessageDriven(activationConfig = {
> @ActivationConfigProperty(propertyName = "destinationType",
> propertyValue = "javax.jms.Queue"),
> @ActivationConfigProperty(propertyName = "destination",
> propertyValue = "JobQueue"),
> @ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
> propertyValue = "1"),
> @ActivationConfigProperty(propertyName = "maxSessions",
> propertyValue = "200"),
> @ActivationConfigProperty(propertyName = "acknowledgeMode",
> propertyValue = "Auto-acknowledge") })
>
> =========================================
>
> With this configuration, I am still facing some interruptions.
>
> Sometimes, TomEE simply stops consuming the queue (once or twice per week),
> so I have to restart it. I am still checking the logs for anything
> suspicious.
>
> I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone AMQ
> 5.10 stable in a linux environment and oracle JDK 7.
>
> But I'd like some advice on how to properly set up activemq / tomee in this
> scenario (that's why I am sending this email to both activemq and tomee
> lists)
>
> Thank you very much, any help is welcome.
>
> []
>
> Leo
>