You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ncsridhar <sr...@uc4.com> on 2010/12/02 19:00:12 UTC
Topic messages lost even when they are persistent - help!
Hi,
I am a new user trying to use activemq for my ESB. I am trying to use the
sample programs TopicPublisher and TopicLIstener. I am posting messages to
ActiveMQ topics called "topictest.messages" , "topictest.control". I am
marking them as persistent in the following ways:
>>>>>>>>>>
ActiveMQConnectionFactory factory = new
ActiveMQConnectionFactory(url);
connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
//session = connection.createSession(false,
Session.SESSION_TRANSACTED);
topic = session.createTopic("topictest.messages");
control = session.createTopic("topictest.control");
publisher = session.createProducer(topic);
publisher.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>>>>>>> and also when sending the actual message itself >>>>>>>>>
for (int i = 0; i < messages; i++) {
publisher.send(msg, DeliveryMode.PERSISTENT, 3, 1);
if ((i + 1) % 1000 == 0) {
System.out.println("Sent " + (i + 1) + " messages");
}
}
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
I have created a durable subscriber on the topictest.messages topic. When I
run the publisher program it posts the messages fine and I can verify that,
but they dont survive a restart. I cant see what I am doing wrong. The odd
behavior is that when I put messages on my queues, they survive restarts by
default, but same is not true for the topics. Below is my activemq.xml:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this
configuration file -->
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.base}/conf/credentials.properties</value>
</property>
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="localhost" dataDirectory="${activemq.base}/data"
persistent="true" useShutdownHook="false">
<!--
For better performances use VM cursor and small memory limit.
For more information, see:
http://activemq.apache.org/message-cursors.html
Also, if your producer is "hanging", it's probably due to
producer flow control.
For more information, see:
http://activemq.apache.org/producer-flow-control.html
-->
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true"
memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true"
memoryLimit="1mb">
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is
exposed in
JMX. By default, ActiveMQ uses the MBean server that is started
by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<!--
Configure message persistence for the broker. The default
persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
-->
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb" />
</persistenceAdapter>
<!--
The systemUsage controls the maximum amount of space the broker
will
use before slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="20 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="100 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
-->
<!--
The transport connectors expose ActiveMQ over a given protocol
to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
</transportConnectors>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
It also includes Camel (with its web console), see
${ACTIVEMQ_HOME}/conf/camel.xml for more info
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
<import resource="camel.xml"/>
</beans>
--
View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3069828.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Topic messages lost even when they are persistent - help!
Posted by ncsridhar <sr...@uc4.com>.
Yes, you are right and thanks!
But it is disappointing that when every other JMS server outthere has a
decent Admin portal, this thing kind of sucks. Do you know of any other 3rd
party softwares that I can use to monitor and manage ActiveMQ?
Thanks.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3072596.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Topic messages lost even when they are persistent - help!
Posted by Stan Lewis <sl...@fusesource.com>.
Yeah, just tried it here and the counter on the topic is misleading.
I believe these are in-memory only counters actually, however if you
take a look at the subscription after a restart you'll see an enqueued
message and you can dequeue it using a real consumer such as the
example consumer in the "examples" directory.
On Fri, Dec 3, 2010 at 1:43 PM, ncsridhar <sr...@uc4.com> wrote:
>
> I tried what you said, but it didn't solve the issue. Here are the steps
> again:
>
>
> - I started activemq
> - created a brand new topic and a durable subscriber on it, I did both of
> those actions using the admin console gui on localhost:8161
> - put some "persistent" messages on the topic using the admin gui again. At
> this point, I can verify the "No. of consumers" to be 1 and the "Messages
> Enqueued" as 1 and "Message Dequeued" as 0.
> - restart the activemq using the command line tool, activemq-admin.bat stop
> for stopping, followed by activemq.bat for starting
> - The message on the topic are lost, again!!
>
> In this exercise, I am not running any java programs or anything. Purely
> using the admin console. Let me know what am I doing wrong. I worked on
> WebsphereMQ quite a bit in the past and the same sscenario would keep the
> messages there.
>
> Thanks.
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3071559.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Stan Lewis
FuseSource
Email: slewis@fusesource.com
Web: http://fusesource.com
Twitter: gashcrumb
Re: Topic messages lost even when they are persistent - help!
Posted by ncsridhar <sr...@uc4.com>.
I tried what you said, but it didn't solve the issue. Here are the steps
again:
- I started activemq
- created a brand new topic and a durable subscriber on it, I did both of
those actions using the admin console gui on localhost:8161
- put some "persistent" messages on the topic using the admin gui again. At
this point, I can verify the "No. of consumers" to be 1 and the "Messages
Enqueued" as 1 and "Message Dequeued" as 0.
- restart the activemq using the command line tool, activemq-admin.bat stop
for stopping, followed by activemq.bat for starting
- The message on the topic are lost, again!!
In this exercise, I am not running any java programs or anything. Purely
using the admin console. Let me know what am I doing wrong. I worked on
WebsphereMQ quite a bit in the past and the same sscenario would keep the
messages there.
Thanks.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3071559.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Topic messages lost even when they are persistent - help!
Posted by Stan Lewis <sl...@fusesource.com>.
You need to create the durable subscription *before* sending messages
to the topic, otherwise the broker will drop them regardless if the
messages are marked as persistent or not. The only way a topic will
store messages is via a durable subscription.
On Fri, Dec 3, 2010 at 10:36 AM, ncsridhar <sr...@uc4.com> wrote:
>
> I actually did not run the consumer atleast once on this topic at all. Here
> is what I did:
>
> - I ran a publisher program that put about 10K PERSISTENT msgs on the topic
> - I verified that all 10K msgs are on the topic using admin console
> - I used the activemq admin console that is available at
> http://localhost:8161/admin/index.jsp to create the durable subscriber on
> that topic
> - I restarted the activemq broker and discovered that all messages are lost,
> but the topic itself is there
>
> Where as I put a persistent msg in a "queue" sometime ago and I restarted
> several times after that and the message is still there. So I am not
> understanding where is this going wrong.
>
>
>
> Thanks.
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3071231.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Stan Lewis
FuseSource
Email: slewis@fusesource.com
Web: http://fusesource.com
Twitter: gashcrumb
Re: Topic messages lost even when they are persistent - help!
Posted by ncsridhar <sr...@uc4.com>.
I actually did not run the consumer atleast once on this topic at all. Here
is what I did:
- I ran a publisher program that put about 10K PERSISTENT msgs on the topic
- I verified that all 10K msgs are on the topic using admin console
- I used the activemq admin console that is available at
http://localhost:8161/admin/index.jsp to create the durable subscriber on
that topic
- I restarted the activemq broker and discovered that all messages are lost,
but the topic itself is there
Where as I put a persistent msg in a "queue" sometime ago and I restarted
several times after that and the message is still there. So I am not
understanding where is this going wrong.
Thanks.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3071231.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Topic messages lost even when they are persistent - help!
Posted by Gary Tully <ga...@gmail.com>.
can you post your consumer, are you specifying the same connection and
subscription id on reconnect?
On 2 December 2010 18:00, ncsridhar <sr...@uc4.com> wrote:
>
> Hi,
>
> I am a new user trying to use activemq for my ESB. I am trying to use the
> sample programs TopicPublisher and TopicLIstener. I am posting messages to
> ActiveMQ topics called "topictest.messages" , "topictest.control". I am
> marking them as persistent in the following ways:
>
>>>>>>>>>>>
> ActiveMQConnectionFactory factory = new
> ActiveMQConnectionFactory(url);
> connection = factory.createConnection();
> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
> //session = connection.createSession(false,
> Session.SESSION_TRANSACTED);
> topic = session.createTopic("topictest.messages");
> control = session.createTopic("topictest.control");
>
> publisher = session.createProducer(topic);
> publisher.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>>>>>>>> and also when sending the actual message itself >>>>>>>>>
> for (int i = 0; i < messages; i++) {
> publisher.send(msg, DeliveryMode.PERSISTENT, 3, 1);
> if ((i + 1) % 1000 == 0) {
> System.out.println("Sent " + (i + 1) + " messages");
> }
> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>
> I have created a durable subscriber on the topictest.messages topic. When I
> run the publisher program it posts the messages fine and I can verify that,
> but they dont survive a restart. I cant see what I am doing wrong. The odd
> behavior is that when I put messages on my queues, they survive restarts by
> default, but same is not true for the topics. Below is my activemq.xml:
>
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:amq="http://activemq.apache.org/schema/core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core.xsd">
>
> <!-- Allows us to use system properties as variables in this
> configuration file -->
> <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="locations">
> <value>file:${activemq.base}/conf/credentials.properties</value>
> </property>
> </bean>
>
> <!--
> The <broker> element is used to configure the ActiveMQ broker.
> -->
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.base}/data"
> persistent="true" useShutdownHook="false">
>
> <!--
> For better performances use VM cursor and small memory limit.
> For more information, see:
>
> http://activemq.apache.org/message-cursors.html
>
> Also, if your producer is "hanging", it's probably due to
> producer flow control.
> For more information, see:
> http://activemq.apache.org/producer-flow-control.html
> -->
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="1mb">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb">
> <!-- Use VM cursor for better latency
> For more information, see:
>
> http://activemq.apache.org/message-cursors.html
>
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> -->
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
>
> <!--
> The managementContext is used to configure how ActiveMQ is
> exposed in
> JMX. By default, ActiveMQ uses the MBean server that is started
> by
> the JVM. For more information, see:
>
> http://activemq.apache.org/jmx.html
> -->
> <managementContext>
> <managementContext createConnector="false"/>
> </managementContext>
>
> <!--
> Configure message persistence for the broker. The default
> persistence
> mechanism is the KahaDB store (identified by the kahaDB tag).
> For more information, see:
>
> http://activemq.apache.org/persistence.html
> -->
> <persistenceAdapter>
> <kahaDB directory="${activemq.base}/data/kahadb" />
> </persistenceAdapter>
>
>
> <!--
> The systemUsage controls the maximum amount of space the broker
> will
> use before slowing down producers. For more information, see:
>
> http://activemq.apache.org/producer-flow-control.html
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="20 mb"/>
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="1 gb"/>
> </storeUsage>
> <tempUsage>
> <tempUsage limit="100 mb"/>
> </tempUsage>
> </systemUsage>
> </systemUsage>
> -->
>
> <!--
> The transport connectors expose ActiveMQ over a given protocol
> to
> clients and other brokers. For more information, see:
>
> http://activemq.apache.org/configuring-transports.html
> -->
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> </transportConnectors>
>
> </broker>
>
> <!--
> Enable web consoles, REST and Ajax APIs and demos
> It also includes Camel (with its web console), see
> ${ACTIVEMQ_HOME}/conf/camel.xml for more info
>
> Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
> -->
> <import resource="jetty.xml"/>
> <import resource="camel.xml"/>
>
> </beans>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Topic-messages-lost-even-when-they-are-persistent-help-tp3069828p3069828.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
http://blog.garytully.com
http://fusesource.com