You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Maarten_D <ma...@gmail.com> on 2010/02/16 17:11:01 UTC
Re: Slow sending of messages
Hi Adrian,
Have you made any progress on this issue? I believe I'm running into the
same things occasionally.
Regards,
Maarten
Adrian A wrote:
>
> Setting them to MapMessages (what I am using) makes it even worse:
>
> //TextMessage message = session.createTextMessage(createMessageText(i));
> MapMessage message = session.createMapMessage();
> message.setJMSType("blahasdadasdsd");
> message.setLong("test1",2222);
> message.setLong("test2",2222);
> message.setLong("test3",2222);
> message.setLong("test4",2222);
> message.setLong("test5",2222);
>
> by 10 to 20 times worse.
>
>
> Adrian A wrote:
>>
>> long start = System.currentTimeMillis();
>> producer.send(message);
>> long stop = System.currentTimeMillis();
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27610689.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
I should mention that I turned logging up to DEBUG, like Adrian said, and at
the point that the broker gets clobbered the topic memory is filled to 100%
and I get a lot of messages like the ones below:
2010-02-17 16:00:05,866 [DEBUG] org.apache.activemq.usage.Usage - Memory
usage change. from: 100, to: 99
2010-02-17 16:00:05,866 [DEBUG] org.apache.activemq.usage.Usage - Memory
usage change. from: 99, to: 100
Maarten_D wrote:
>
> No problem:
>
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:amq="http://activemq.apache.org/schema/core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core.xsd
> http://mortbay.com/schemas/jetty/1.0
> http://jetty.mortbay.org/jetty.xsd">
>
> <!-- Allows us to use system properties as variables in this
> configuration file -->
> <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="location" value="file:/var/amq/broker.properties" />
> </bean>
>
> <!--
>
> ******************************************************************************************************************
> ** ActiveMQ broker
>
> ******************************************************************************************************************
> -->
> <broker id="broker" useJmx="true" brokerName="broker" start="true"
> xmlns="http://activemq.apache.org/schema/core"
> dataDirectory="/var/amq" advisorySupport="false"
> persistenceAdapter="#store">
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry queue=">" memoryLimit="64 mb"
> producerFlowControl="false" />
> <policyEntry topic=">" memoryLimit="64 mb"
> producerFlowControl="true" />
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <managementContext>
> <managementContext useMBeanServer="true"
> jmxDomainName="org.apache.activemq"
> createMBeanServer="true"
> createConnector="false"
> connectorPort="1100"
> connectorPath="/jmxrmi"/>
> </managementContext>
>
> <persistenceAdapter id="store">
> <kahaDB enableJournalDiskSyncs="false"
> journalMaxFileLength="32mb"
> enableIndexWriteAsync="true"
> directory="/var/amq/broker"
> indexWriteBatchSize="1000" />
> </persistenceAdapter>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="512 mb" />
> </memoryUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="cearchive" uri="tcp://0.0.0.0:61616" />
> </transportConnectors>
> </broker>
>
> <!-- Here we start an embedded webserver for the admin console -->
> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
> <connectors>
> <nioConnector port="8161"/>
> </connectors>
> <handlers>
> <webAppContext contextPath="/admin"
> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/>
> </handlers>
> </jetty>
> </beans>
>
>
>
> rajdavies wrote:
>>
>> can you send your broker config ?
>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>
>>>
>>> The topics and queues are filled using a Spring JMSTemplate that has
>>> it's own
>>> connection factory, and dequeuing is done by message listeners that
>>> all have
>>> their own connection. So everything should have its own connection,
>>> let
>>> alone session.
>>>
>>> I'll do another run on debug and see what it turns up.
>>>
>>>
>>> Adrian A wrote:
>>>>
>>>> you are running separate sessions for each of those dequeue/enqueue
>>>> stats?
>>>>
>>>> in my flow control tests even when one particular session was hung
>>>> other
>>>> sessions to the same broker was fine, just when I overwhelmed
>>>> broker and
>>>> GC / disk checkpointing occurred that it got really bad.
>>>>
>>>> have you turned on debugging as that although verbose is a wealth of
>>>> information!
>>>>
>>>>
>>>>
>>>> Maarten_D wrote:
>>>>>
>>>>> Hi Adrian, thanks for your response.
>>>>>
>>>>> I'm currently running tests where I have a very fast producer and a
>>>>> relatively slow consumer. The producer publishes persistent
>>>>> messages to a
>>>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>>>> to a
>>>>> difference of around 80,000 messages. The producer then gets
>>>>> whacked and
>>>>> the enqueue graph in visualvm completely levels off. This is more
>>>>> or less
>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>> However, the
>>>>> entire broker stalls. I have several queues that are filled and
>>>>> emptied
>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>> flatline
>>>>> as well, even though flow control shouldn't apply to them. Thats
>>>>> why I
>>>>> was interested to find out if you'd discovered some kind of fresh
>>>>> angle.
>>>>> Regards,
>>>>> Maarten
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624994.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
Am doing a run with the 5.4 snapshot and your suggested paramter now, but the
behaviour is exactly the same.
Regards,
Maarten
rajdavies wrote:
>
> The apache one :)
> On 17 Feb 2010, at 17:41, Maarten_D wrote:
>
>>
>> Do you mean activemq or fuse MB? Cause I can't find the 5.4 snapshot
>> of fuse
>> MB (I looked here
>> http://repo.fusesource.com/maven2-snapshot/com/iona/fuse/fuse-message-broker/)
>> .
>>
>>
>> rajdavies wrote:
>>>
>>> damn - can you try 5.4-SNAPSHOT ?
>>> On 17 Feb 2010, at 15:33, Maarten_D wrote:
>>>
>>>>
>>>> Hang on, I was a bit premature in sending that last message: I'm
>>>> actually
>>>> using Fuse 5.3.0.5 and its version of KahaDB doesn't support the
>>>> indexCacheSize parameter :S
>>>>
>>>>
>>>> Maarten_D wrote:
>>>>>
>>>>> Sure, I'll try that now. In the mean time, could you perhaps tell
>>>>> me why
>>>>> (and how) that would help?
>>>>> Thanks in advance,
>>>>> Maarten
>>>>>
>>>>>
>>>>> rajdavies wrote:
>>>>>>
>>>>>> Can you try increasing the cache size for KahaDB - to 10000 ? -
>>>>>> see
>>>>>> http://activemq.apache.org/kahadb.html
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> Rob
>>>>>> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>>>>>>
>>>>>>>
>>>>>>> No problem:
>>>>>>>
>>>>>>> <beans
>>>>>>> xmlns="http://www.springframework.org/schema/beans"
>>>>>>> xmlns:amq="http://activemq.apache.org/schema/core"
>>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>>>>>> http://activemq.apache.org/schema/core
>>>>>>> http://activemq.apache.org/schema/core/activemq-core.xsd
>>>>>>> http://mortbay.com/schemas/jetty/1.0
>>>>>>> http://jetty.mortbay.org/jetty.xsd">
>>>>>>>
>>>>>>> <!-- Allows us to use system properties as variables in this
>>>>>>> configuration
>>>>>>> file -->
>>>>>>> <bean
>>>>>>> class
>>>>>>> =
>>>>>>> "org
>>>>>>> .springframework
>>>>>>> .beans.factory.config.PropertyPlaceholderConfigurer">
>>>>>>> <property name="location" value="file:/var/amq/
>>>>>>> broker.properties" />
>>>>>>> </bean>
>>>>>>>
>>>>>>> <!--
>>>>>>>
>>>>>>> ******************************************************************************************************************
>>>>>>> ** ActiveMQ broker
>>>>>>>
>>>>>>> ******************************************************************************************************************
>>>>>>> -->
>>>>>>> <broker id="broker" useJmx="true" brokerName="broker"
>>>>>>> start="true"
>>>>>>> xmlns="http://activemq.apache.org/schema/core"
>>>>>>> dataDirectory="/var/amq" advisorySupport="false"
>>>>>>> persistenceAdapter="#store">
>>>>>>>
>>>>>>> <destinationPolicy>
>>>>>>> <policyMap>
>>>>>>> <policyEntries>
>>>>>>> <policyEntry queue=">" memoryLimit="64 mb"
>>>>>>> producerFlowControl="false" />
>>>>>>> <policyEntry topic=">" memoryLimit="64 mb"
>>>>>>> producerFlowControl="true" />
>>>>>>> </policyEntries>
>>>>>>> </policyMap>
>>>>>>> </destinationPolicy>
>>>>>>>
>>>>>>> <managementContext>
>>>>>>> <managementContext useMBeanServer="true"
>>>>>>> jmxDomainName="org.apache.activemq"
>>>>>>> createMBeanServer="true"
>>>>>>> createConnector="false"
>>>>>>> connectorPort="1100"
>>>>>>> connectorPath="/jmxrmi"/>
>>>>>>> </managementContext>
>>>>>>>
>>>>>>> <persistenceAdapter id="store">
>>>>>>> <kahaDB enableJournalDiskSyncs="false"
>>>>>>> journalMaxFileLength="32mb"
>>>>>>> enableIndexWriteAsync="true"
>>>>>>> directory="/var/amq/broker"
>>>>>>> indexWriteBatchSize="1000" />
>>>>>>> </persistenceAdapter>
>>>>>>>
>>>>>>> <systemUsage>
>>>>>>> <systemUsage>
>>>>>>> <memoryUsage>
>>>>>>> <memoryUsage limit="512 mb" />
>>>>>>> </memoryUsage>
>>>>>>> </systemUsage>
>>>>>>> </systemUsage>
>>>>>>>
>>>>>>> <transportConnectors>
>>>>>>> <transportConnector name="cearchive" uri="tcp://
>>>>>>> 0.0.0.0:61616" />
>>>>>>> </transportConnectors>
>>>>>>> </broker>
>>>>>>>
>>>>>>> <!-- Here we start an embedded webserver for the admin console
>>>>>>> -->
>>>>>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>>>>>>> <connectors>
>>>>>>> <nioConnector port="8161"/>
>>>>>>> </connectors>
>>>>>>> <handlers>
>>>>>>> <webAppContext contextPath="/admin"
>>>>>>> resourceBase="${activemq.base}/webapps/admin"
>>>>>>> logUrlOnStart="true"/>
>>>>>>> </handlers>
>>>>>>> </jetty>
>>>>>>> </beans>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> rajdavies wrote:
>>>>>>>>
>>>>>>>> can you send your broker config ?
>>>>>>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> The topics and queues are filled using a Spring JMSTemplate
>>>>>>>>> that has
>>>>>>>>> it's own
>>>>>>>>> connection factory, and dequeuing is done by message listeners
>>>>>>>>> that
>>>>>>>>> all have
>>>>>>>>> their own connection. So everything should have its own
>>>>>>>>> connection,
>>>>>>>>> let
>>>>>>>>> alone session.
>>>>>>>>>
>>>>>>>>> I'll do another run on debug and see what it turns up.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Adrian A wrote:
>>>>>>>>>>
>>>>>>>>>> you are running separate sessions for each of those dequeue/
>>>>>>>>>> enqueue
>>>>>>>>>> stats?
>>>>>>>>>>
>>>>>>>>>> in my flow control tests even when one particular session was
>>>>>>>>>> hung
>>>>>>>>>> other
>>>>>>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>>>>>>> broker and
>>>>>>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>>>>>>
>>>>>>>>>> have you turned on debugging as that although verbose is a
>>>>>>>>>> wealth
>>>>>>>>>> of
>>>>>>>>>> information!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Maarten_D wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Adrian, thanks for your response.
>>>>>>>>>>>
>>>>>>>>>>> I'm currently running tests where I have a very fast producer
>>>>>>>>>>> and a
>>>>>>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>>>>>>> messages to a
>>>>>>>>>>> topic, where the enqueue and dequeue count diverge fairly
>>>>>>>>>>> rapidly
>>>>>>>>>>> to a
>>>>>>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>>>>>>> whacked and
>>>>>>>>>>> the enqueue graph in visualvm completely levels off. This is
>>>>>>>>>>> more
>>>>>>>>>>> or less
>>>>>>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>>>>>>> However, the
>>>>>>>>>>> entire broker stalls. I have several queues that are filled
>>>>>>>>>>> and
>>>>>>>>>>> emptied
>>>>>>>>>>> at the same time as the topic, and their dequeue/enqueue
>>>>>>>>>>> stats
>>>>>>>>>>> flatline
>>>>>>>>>>> as well, even though flow control shouldn't apply to them.
>>>>>>>>>>> Thats
>>>>>>>>>>> why I
>>>>>>>>>>> was interested to find out if you'd discovered some kind of
>>>>>>>>>>> fresh
>>>>>>>>>>> angle.
>>>>>>>>>>> Regards,
>>>>>>>>>>> Maarten
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>>>>>>> Sent from the ActiveMQ - User mailing list archive at
>>>>>>>>> Nabble.com.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Rob Davies
>>>>>>>> http://twitter.com/rajdavies
>>>>>>>> I work here: http://fusesource.com
>>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>
>>>>>>
>>>>>> Rob Davies
>>>>>> http://twitter.com/rajdavies
>>>>>> I work here: http://fusesource.com
>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625537.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>
>>> Rob Davies
>>> http://twitter.com/rajdavies
>>> I work here: http://fusesource.com
>>> My Blog: http://rajdavies.blogspot.com/
>>> I'm writing this: http://www.manning.com/snyder/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27627497.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27637242.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Rob Davies <ra...@gmail.com>.
The apache one :)
On 17 Feb 2010, at 17:41, Maarten_D wrote:
>
> Do you mean activemq or fuse MB? Cause I can't find the 5.4 snapshot
> of fuse
> MB (I looked here
> http://repo.fusesource.com/maven2-snapshot/com/iona/fuse/fuse-message-broker/)
> .
>
>
> rajdavies wrote:
>>
>> damn - can you try 5.4-SNAPSHOT ?
>> On 17 Feb 2010, at 15:33, Maarten_D wrote:
>>
>>>
>>> Hang on, I was a bit premature in sending that last message: I'm
>>> actually
>>> using Fuse 5.3.0.5 and its version of KahaDB doesn't support the
>>> indexCacheSize parameter :S
>>>
>>>
>>> Maarten_D wrote:
>>>>
>>>> Sure, I'll try that now. In the mean time, could you perhaps tell
>>>> me why
>>>> (and how) that would help?
>>>> Thanks in advance,
>>>> Maarten
>>>>
>>>>
>>>> rajdavies wrote:
>>>>>
>>>>> Can you try increasing the cache size for KahaDB - to 10000 ? -
>>>>> see
>>>>> http://activemq.apache.org/kahadb.html
>>>>>
>>>>> cheers,
>>>>>
>>>>> Rob
>>>>> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>>>>>
>>>>>>
>>>>>> No problem:
>>>>>>
>>>>>> <beans
>>>>>> xmlns="http://www.springframework.org/schema/beans"
>>>>>> xmlns:amq="http://activemq.apache.org/schema/core"
>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>>>>> http://activemq.apache.org/schema/core
>>>>>> http://activemq.apache.org/schema/core/activemq-core.xsd
>>>>>> http://mortbay.com/schemas/jetty/1.0
>>>>>> http://jetty.mortbay.org/jetty.xsd">
>>>>>>
>>>>>> <!-- Allows us to use system properties as variables in this
>>>>>> configuration
>>>>>> file -->
>>>>>> <bean
>>>>>> class
>>>>>> =
>>>>>> "org
>>>>>> .springframework
>>>>>> .beans.factory.config.PropertyPlaceholderConfigurer">
>>>>>> <property name="location" value="file:/var/amq/
>>>>>> broker.properties" />
>>>>>> </bean>
>>>>>>
>>>>>> <!--
>>>>>>
>>>>>> ******************************************************************************************************************
>>>>>> ** ActiveMQ broker
>>>>>>
>>>>>> ******************************************************************************************************************
>>>>>> -->
>>>>>> <broker id="broker" useJmx="true" brokerName="broker"
>>>>>> start="true"
>>>>>> xmlns="http://activemq.apache.org/schema/core"
>>>>>> dataDirectory="/var/amq" advisorySupport="false"
>>>>>> persistenceAdapter="#store">
>>>>>>
>>>>>> <destinationPolicy>
>>>>>> <policyMap>
>>>>>> <policyEntries>
>>>>>> <policyEntry queue=">" memoryLimit="64 mb"
>>>>>> producerFlowControl="false" />
>>>>>> <policyEntry topic=">" memoryLimit="64 mb"
>>>>>> producerFlowControl="true" />
>>>>>> </policyEntries>
>>>>>> </policyMap>
>>>>>> </destinationPolicy>
>>>>>>
>>>>>> <managementContext>
>>>>>> <managementContext useMBeanServer="true"
>>>>>> jmxDomainName="org.apache.activemq"
>>>>>> createMBeanServer="true"
>>>>>> createConnector="false"
>>>>>> connectorPort="1100"
>>>>>> connectorPath="/jmxrmi"/>
>>>>>> </managementContext>
>>>>>>
>>>>>> <persistenceAdapter id="store">
>>>>>> <kahaDB enableJournalDiskSyncs="false"
>>>>>> journalMaxFileLength="32mb"
>>>>>> enableIndexWriteAsync="true"
>>>>>> directory="/var/amq/broker"
>>>>>> indexWriteBatchSize="1000" />
>>>>>> </persistenceAdapter>
>>>>>>
>>>>>> <systemUsage>
>>>>>> <systemUsage>
>>>>>> <memoryUsage>
>>>>>> <memoryUsage limit="512 mb" />
>>>>>> </memoryUsage>
>>>>>> </systemUsage>
>>>>>> </systemUsage>
>>>>>>
>>>>>> <transportConnectors>
>>>>>> <transportConnector name="cearchive" uri="tcp://
>>>>>> 0.0.0.0:61616" />
>>>>>> </transportConnectors>
>>>>>> </broker>
>>>>>>
>>>>>> <!-- Here we start an embedded webserver for the admin console
>>>>>> -->
>>>>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>>>>>> <connectors>
>>>>>> <nioConnector port="8161"/>
>>>>>> </connectors>
>>>>>> <handlers>
>>>>>> <webAppContext contextPath="/admin"
>>>>>> resourceBase="${activemq.base}/webapps/admin"
>>>>>> logUrlOnStart="true"/>
>>>>>> </handlers>
>>>>>> </jetty>
>>>>>> </beans>
>>>>>>
>>>>>>
>>>>>>
>>>>>> rajdavies wrote:
>>>>>>>
>>>>>>> can you send your broker config ?
>>>>>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> The topics and queues are filled using a Spring JMSTemplate
>>>>>>>> that has
>>>>>>>> it's own
>>>>>>>> connection factory, and dequeuing is done by message listeners
>>>>>>>> that
>>>>>>>> all have
>>>>>>>> their own connection. So everything should have its own
>>>>>>>> connection,
>>>>>>>> let
>>>>>>>> alone session.
>>>>>>>>
>>>>>>>> I'll do another run on debug and see what it turns up.
>>>>>>>>
>>>>>>>>
>>>>>>>> Adrian A wrote:
>>>>>>>>>
>>>>>>>>> you are running separate sessions for each of those dequeue/
>>>>>>>>> enqueue
>>>>>>>>> stats?
>>>>>>>>>
>>>>>>>>> in my flow control tests even when one particular session was
>>>>>>>>> hung
>>>>>>>>> other
>>>>>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>>>>>> broker and
>>>>>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>>>>>
>>>>>>>>> have you turned on debugging as that although verbose is a
>>>>>>>>> wealth
>>>>>>>>> of
>>>>>>>>> information!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Maarten_D wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Adrian, thanks for your response.
>>>>>>>>>>
>>>>>>>>>> I'm currently running tests where I have a very fast producer
>>>>>>>>>> and a
>>>>>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>>>>>> messages to a
>>>>>>>>>> topic, where the enqueue and dequeue count diverge fairly
>>>>>>>>>> rapidly
>>>>>>>>>> to a
>>>>>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>>>>>> whacked and
>>>>>>>>>> the enqueue graph in visualvm completely levels off. This is
>>>>>>>>>> more
>>>>>>>>>> or less
>>>>>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>>>>>> However, the
>>>>>>>>>> entire broker stalls. I have several queues that are filled
>>>>>>>>>> and
>>>>>>>>>> emptied
>>>>>>>>>> at the same time as the topic, and their dequeue/enqueue
>>>>>>>>>> stats
>>>>>>>>>> flatline
>>>>>>>>>> as well, even though flow control shouldn't apply to them.
>>>>>>>>>> Thats
>>>>>>>>>> why I
>>>>>>>>>> was interested to find out if you'd discovered some kind of
>>>>>>>>>> fresh
>>>>>>>>>> angle.
>>>>>>>>>> Regards,
>>>>>>>>>> Maarten
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>>>>>> Sent from the ActiveMQ - User mailing list archive at
>>>>>>>> Nabble.com.
>>>>>>>>
>>>>>>>
>>>>>>> Rob Davies
>>>>>>> http://twitter.com/rajdavies
>>>>>>> I work here: http://fusesource.com
>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>
>>>>>
>>>>> Rob Davies
>>>>> http://twitter.com/rajdavies
>>>>> I work here: http://fusesource.com
>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625537.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27627497.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
Do you mean activemq or fuse MB? Cause I can't find the 5.4 snapshot of fuse
MB (I looked here
http://repo.fusesource.com/maven2-snapshot/com/iona/fuse/fuse-message-broker/).
rajdavies wrote:
>
> damn - can you try 5.4-SNAPSHOT ?
> On 17 Feb 2010, at 15:33, Maarten_D wrote:
>
>>
>> Hang on, I was a bit premature in sending that last message: I'm
>> actually
>> using Fuse 5.3.0.5 and its version of KahaDB doesn't support the
>> indexCacheSize parameter :S
>>
>>
>> Maarten_D wrote:
>>>
>>> Sure, I'll try that now. In the mean time, could you perhaps tell
>>> me why
>>> (and how) that would help?
>>> Thanks in advance,
>>> Maarten
>>>
>>>
>>> rajdavies wrote:
>>>>
>>>> Can you try increasing the cache size for KahaDB - to 10000 ? - see
>>>> http://activemq.apache.org/kahadb.html
>>>>
>>>> cheers,
>>>>
>>>> Rob
>>>> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>>>>
>>>>>
>>>>> No problem:
>>>>>
>>>>> <beans
>>>>> xmlns="http://www.springframework.org/schema/beans"
>>>>> xmlns:amq="http://activemq.apache.org/schema/core"
>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>>>> http://activemq.apache.org/schema/core
>>>>> http://activemq.apache.org/schema/core/activemq-core.xsd
>>>>> http://mortbay.com/schemas/jetty/1.0
>>>>> http://jetty.mortbay.org/jetty.xsd">
>>>>>
>>>>> <!-- Allows us to use system properties as variables in this
>>>>> configuration
>>>>> file -->
>>>>> <bean
>>>>> class
>>>>> =
>>>>> "org
>>>>> .springframework
>>>>> .beans.factory.config.PropertyPlaceholderConfigurer">
>>>>> <property name="location" value="file:/var/amq/
>>>>> broker.properties" />
>>>>> </bean>
>>>>>
>>>>> <!--
>>>>>
>>>>> ******************************************************************************************************************
>>>>> ** ActiveMQ broker
>>>>>
>>>>> ******************************************************************************************************************
>>>>> -->
>>>>> <broker id="broker" useJmx="true" brokerName="broker" start="true"
>>>>> xmlns="http://activemq.apache.org/schema/core"
>>>>> dataDirectory="/var/amq" advisorySupport="false"
>>>>> persistenceAdapter="#store">
>>>>>
>>>>> <destinationPolicy>
>>>>> <policyMap>
>>>>> <policyEntries>
>>>>> <policyEntry queue=">" memoryLimit="64 mb"
>>>>> producerFlowControl="false" />
>>>>> <policyEntry topic=">" memoryLimit="64 mb"
>>>>> producerFlowControl="true" />
>>>>> </policyEntries>
>>>>> </policyMap>
>>>>> </destinationPolicy>
>>>>>
>>>>> <managementContext>
>>>>> <managementContext useMBeanServer="true"
>>>>> jmxDomainName="org.apache.activemq"
>>>>> createMBeanServer="true"
>>>>> createConnector="false"
>>>>> connectorPort="1100"
>>>>> connectorPath="/jmxrmi"/>
>>>>> </managementContext>
>>>>>
>>>>> <persistenceAdapter id="store">
>>>>> <kahaDB enableJournalDiskSyncs="false"
>>>>> journalMaxFileLength="32mb"
>>>>> enableIndexWriteAsync="true"
>>>>> directory="/var/amq/broker"
>>>>> indexWriteBatchSize="1000" />
>>>>> </persistenceAdapter>
>>>>>
>>>>> <systemUsage>
>>>>> <systemUsage>
>>>>> <memoryUsage>
>>>>> <memoryUsage limit="512 mb" />
>>>>> </memoryUsage>
>>>>> </systemUsage>
>>>>> </systemUsage>
>>>>>
>>>>> <transportConnectors>
>>>>> <transportConnector name="cearchive" uri="tcp://
>>>>> 0.0.0.0:61616" />
>>>>> </transportConnectors>
>>>>> </broker>
>>>>>
>>>>> <!-- Here we start an embedded webserver for the admin console -->
>>>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>>>>> <connectors>
>>>>> <nioConnector port="8161"/>
>>>>> </connectors>
>>>>> <handlers>
>>>>> <webAppContext contextPath="/admin"
>>>>> resourceBase="${activemq.base}/webapps/admin"
>>>>> logUrlOnStart="true"/>
>>>>> </handlers>
>>>>> </jetty>
>>>>> </beans>
>>>>>
>>>>>
>>>>>
>>>>> rajdavies wrote:
>>>>>>
>>>>>> can you send your broker config ?
>>>>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>>>>
>>>>>>>
>>>>>>> The topics and queues are filled using a Spring JMSTemplate
>>>>>>> that has
>>>>>>> it's own
>>>>>>> connection factory, and dequeuing is done by message listeners
>>>>>>> that
>>>>>>> all have
>>>>>>> their own connection. So everything should have its own
>>>>>>> connection,
>>>>>>> let
>>>>>>> alone session.
>>>>>>>
>>>>>>> I'll do another run on debug and see what it turns up.
>>>>>>>
>>>>>>>
>>>>>>> Adrian A wrote:
>>>>>>>>
>>>>>>>> you are running separate sessions for each of those dequeue/
>>>>>>>> enqueue
>>>>>>>> stats?
>>>>>>>>
>>>>>>>> in my flow control tests even when one particular session was
>>>>>>>> hung
>>>>>>>> other
>>>>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>>>>> broker and
>>>>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>>>>
>>>>>>>> have you turned on debugging as that although verbose is a
>>>>>>>> wealth
>>>>>>>> of
>>>>>>>> information!
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Maarten_D wrote:
>>>>>>>>>
>>>>>>>>> Hi Adrian, thanks for your response.
>>>>>>>>>
>>>>>>>>> I'm currently running tests where I have a very fast producer
>>>>>>>>> and a
>>>>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>>>>> messages to a
>>>>>>>>> topic, where the enqueue and dequeue count diverge fairly
>>>>>>>>> rapidly
>>>>>>>>> to a
>>>>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>>>>> whacked and
>>>>>>>>> the enqueue graph in visualvm completely levels off. This is
>>>>>>>>> more
>>>>>>>>> or less
>>>>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>>>>> However, the
>>>>>>>>> entire broker stalls. I have several queues that are filled and
>>>>>>>>> emptied
>>>>>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>>>>>> flatline
>>>>>>>>> as well, even though flow control shouldn't apply to them.
>>>>>>>>> Thats
>>>>>>>>> why I
>>>>>>>>> was interested to find out if you'd discovered some kind of
>>>>>>>>> fresh
>>>>>>>>> angle.
>>>>>>>>> Regards,
>>>>>>>>> Maarten
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>
>>>>>>
>>>>>> Rob Davies
>>>>>> http://twitter.com/rajdavies
>>>>>> I work here: http://fusesource.com
>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>
>>>>
>>>> Rob Davies
>>>> http://twitter.com/rajdavies
>>>> I work here: http://fusesource.com
>>>> My Blog: http://rajdavies.blogspot.com/
>>>> I'm writing this: http://www.manning.com/snyder/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625537.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27627497.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Rob Davies <ra...@gmail.com>.
damn - can you try 5.4-SNAPSHOT ?
On 17 Feb 2010, at 15:33, Maarten_D wrote:
>
> Hang on, I was a bit premature in sending that last message: I'm
> actually
> using Fuse 5.3.0.5 and its version of KahaDB doesn't support the
> indexCacheSize parameter :S
>
>
> Maarten_D wrote:
>>
>> Sure, I'll try that now. In the mean time, could you perhaps tell
>> me why
>> (and how) that would help?
>> Thanks in advance,
>> Maarten
>>
>>
>> rajdavies wrote:
>>>
>>> Can you try increasing the cache size for KahaDB - to 10000 ? - see
>>> http://activemq.apache.org/kahadb.html
>>>
>>> cheers,
>>>
>>> Rob
>>> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>>>
>>>>
>>>> No problem:
>>>>
>>>> <beans
>>>> xmlns="http://www.springframework.org/schema/beans"
>>>> xmlns:amq="http://activemq.apache.org/schema/core"
>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>>> http://activemq.apache.org/schema/core
>>>> http://activemq.apache.org/schema/core/activemq-core.xsd
>>>> http://mortbay.com/schemas/jetty/1.0
>>>> http://jetty.mortbay.org/jetty.xsd">
>>>>
>>>> <!-- Allows us to use system properties as variables in this
>>>> configuration
>>>> file -->
>>>> <bean
>>>> class
>>>> =
>>>> "org
>>>> .springframework
>>>> .beans.factory.config.PropertyPlaceholderConfigurer">
>>>> <property name="location" value="file:/var/amq/
>>>> broker.properties" />
>>>> </bean>
>>>>
>>>> <!--
>>>>
>>>> ******************************************************************************************************************
>>>> ** ActiveMQ broker
>>>>
>>>> ******************************************************************************************************************
>>>> -->
>>>> <broker id="broker" useJmx="true" brokerName="broker" start="true"
>>>> xmlns="http://activemq.apache.org/schema/core"
>>>> dataDirectory="/var/amq" advisorySupport="false"
>>>> persistenceAdapter="#store">
>>>>
>>>> <destinationPolicy>
>>>> <policyMap>
>>>> <policyEntries>
>>>> <policyEntry queue=">" memoryLimit="64 mb"
>>>> producerFlowControl="false" />
>>>> <policyEntry topic=">" memoryLimit="64 mb"
>>>> producerFlowControl="true" />
>>>> </policyEntries>
>>>> </policyMap>
>>>> </destinationPolicy>
>>>>
>>>> <managementContext>
>>>> <managementContext useMBeanServer="true"
>>>> jmxDomainName="org.apache.activemq"
>>>> createMBeanServer="true"
>>>> createConnector="false"
>>>> connectorPort="1100"
>>>> connectorPath="/jmxrmi"/>
>>>> </managementContext>
>>>>
>>>> <persistenceAdapter id="store">
>>>> <kahaDB enableJournalDiskSyncs="false"
>>>> journalMaxFileLength="32mb"
>>>> enableIndexWriteAsync="true"
>>>> directory="/var/amq/broker"
>>>> indexWriteBatchSize="1000" />
>>>> </persistenceAdapter>
>>>>
>>>> <systemUsage>
>>>> <systemUsage>
>>>> <memoryUsage>
>>>> <memoryUsage limit="512 mb" />
>>>> </memoryUsage>
>>>> </systemUsage>
>>>> </systemUsage>
>>>>
>>>> <transportConnectors>
>>>> <transportConnector name="cearchive" uri="tcp://
>>>> 0.0.0.0:61616" />
>>>> </transportConnectors>
>>>> </broker>
>>>>
>>>> <!-- Here we start an embedded webserver for the admin console -->
>>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>>>> <connectors>
>>>> <nioConnector port="8161"/>
>>>> </connectors>
>>>> <handlers>
>>>> <webAppContext contextPath="/admin"
>>>> resourceBase="${activemq.base}/webapps/admin"
>>>> logUrlOnStart="true"/>
>>>> </handlers>
>>>> </jetty>
>>>> </beans>
>>>>
>>>>
>>>>
>>>> rajdavies wrote:
>>>>>
>>>>> can you send your broker config ?
>>>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>>>
>>>>>>
>>>>>> The topics and queues are filled using a Spring JMSTemplate
>>>>>> that has
>>>>>> it's own
>>>>>> connection factory, and dequeuing is done by message listeners
>>>>>> that
>>>>>> all have
>>>>>> their own connection. So everything should have its own
>>>>>> connection,
>>>>>> let
>>>>>> alone session.
>>>>>>
>>>>>> I'll do another run on debug and see what it turns up.
>>>>>>
>>>>>>
>>>>>> Adrian A wrote:
>>>>>>>
>>>>>>> you are running separate sessions for each of those dequeue/
>>>>>>> enqueue
>>>>>>> stats?
>>>>>>>
>>>>>>> in my flow control tests even when one particular session was
>>>>>>> hung
>>>>>>> other
>>>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>>>> broker and
>>>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>>>
>>>>>>> have you turned on debugging as that although verbose is a
>>>>>>> wealth
>>>>>>> of
>>>>>>> information!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Maarten_D wrote:
>>>>>>>>
>>>>>>>> Hi Adrian, thanks for your response.
>>>>>>>>
>>>>>>>> I'm currently running tests where I have a very fast producer
>>>>>>>> and a
>>>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>>>> messages to a
>>>>>>>> topic, where the enqueue and dequeue count diverge fairly
>>>>>>>> rapidly
>>>>>>>> to a
>>>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>>>> whacked and
>>>>>>>> the enqueue graph in visualvm completely levels off. This is
>>>>>>>> more
>>>>>>>> or less
>>>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>>>> However, the
>>>>>>>> entire broker stalls. I have several queues that are filled and
>>>>>>>> emptied
>>>>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>>>>> flatline
>>>>>>>> as well, even though flow control shouldn't apply to them.
>>>>>>>> Thats
>>>>>>>> why I
>>>>>>>> was interested to find out if you'd discovered some kind of
>>>>>>>> fresh
>>>>>>>> angle.
>>>>>>>> Regards,
>>>>>>>> Maarten
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>
>>>>>
>>>>> Rob Davies
>>>>> http://twitter.com/rajdavies
>>>>> I work here: http://fusesource.com
>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>
>>> Rob Davies
>>> http://twitter.com/rajdavies
>>> I work here: http://fusesource.com
>>> My Blog: http://rajdavies.blogspot.com/
>>> I'm writing this: http://www.manning.com/snyder/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625537.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
Hang on, I was a bit premature in sending that last message: I'm actually
using Fuse 5.3.0.5 and its version of KahaDB doesn't support the
indexCacheSize parameter :S
Maarten_D wrote:
>
> Sure, I'll try that now. In the mean time, could you perhaps tell me why
> (and how) that would help?
> Thanks in advance,
> Maarten
>
>
> rajdavies wrote:
>>
>> Can you try increasing the cache size for KahaDB - to 10000 ? - see
>> http://activemq.apache.org/kahadb.html
>>
>> cheers,
>>
>> Rob
>> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>>
>>>
>>> No problem:
>>>
>>> <beans
>>> xmlns="http://www.springframework.org/schema/beans"
>>> xmlns:amq="http://activemq.apache.org/schema/core"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>> http://activemq.apache.org/schema/core
>>> http://activemq.apache.org/schema/core/activemq-core.xsd
>>> http://mortbay.com/schemas/jetty/1.0
>>> http://jetty.mortbay.org/jetty.xsd">
>>>
>>> <!-- Allows us to use system properties as variables in this
>>> configuration
>>> file -->
>>> <bean
>>> class
>>> =
>>> "org
>>> .springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>>> <property name="location" value="file:/var/amq/
>>> broker.properties" />
>>> </bean>
>>>
>>> <!--
>>>
>>> ******************************************************************************************************************
>>> ** ActiveMQ broker
>>>
>>> ******************************************************************************************************************
>>> -->
>>> <broker id="broker" useJmx="true" brokerName="broker" start="true"
>>> xmlns="http://activemq.apache.org/schema/core"
>>> dataDirectory="/var/amq" advisorySupport="false"
>>> persistenceAdapter="#store">
>>>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry queue=">" memoryLimit="64 mb"
>>> producerFlowControl="false" />
>>> <policyEntry topic=">" memoryLimit="64 mb"
>>> producerFlowControl="true" />
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>> <managementContext>
>>> <managementContext useMBeanServer="true"
>>> jmxDomainName="org.apache.activemq"
>>> createMBeanServer="true"
>>> createConnector="false"
>>> connectorPort="1100"
>>> connectorPath="/jmxrmi"/>
>>> </managementContext>
>>>
>>> <persistenceAdapter id="store">
>>> <kahaDB enableJournalDiskSyncs="false"
>>> journalMaxFileLength="32mb"
>>> enableIndexWriteAsync="true"
>>> directory="/var/amq/broker"
>>> indexWriteBatchSize="1000" />
>>> </persistenceAdapter>
>>>
>>> <systemUsage>
>>> <systemUsage>
>>> <memoryUsage>
>>> <memoryUsage limit="512 mb" />
>>> </memoryUsage>
>>> </systemUsage>
>>> </systemUsage>
>>>
>>> <transportConnectors>
>>> <transportConnector name="cearchive" uri="tcp://0.0.0.0:61616" />
>>> </transportConnectors>
>>> </broker>
>>>
>>> <!-- Here we start an embedded webserver for the admin console -->
>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>>> <connectors>
>>> <nioConnector port="8161"/>
>>> </connectors>
>>> <handlers>
>>> <webAppContext contextPath="/admin"
>>> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/>
>>> </handlers>
>>> </jetty>
>>> </beans>
>>>
>>>
>>>
>>> rajdavies wrote:
>>>>
>>>> can you send your broker config ?
>>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>>
>>>>>
>>>>> The topics and queues are filled using a Spring JMSTemplate that has
>>>>> it's own
>>>>> connection factory, and dequeuing is done by message listeners that
>>>>> all have
>>>>> their own connection. So everything should have its own connection,
>>>>> let
>>>>> alone session.
>>>>>
>>>>> I'll do another run on debug and see what it turns up.
>>>>>
>>>>>
>>>>> Adrian A wrote:
>>>>>>
>>>>>> you are running separate sessions for each of those dequeue/enqueue
>>>>>> stats?
>>>>>>
>>>>>> in my flow control tests even when one particular session was hung
>>>>>> other
>>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>>> broker and
>>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>>
>>>>>> have you turned on debugging as that although verbose is a wealth
>>>>>> of
>>>>>> information!
>>>>>>
>>>>>>
>>>>>>
>>>>>> Maarten_D wrote:
>>>>>>>
>>>>>>> Hi Adrian, thanks for your response.
>>>>>>>
>>>>>>> I'm currently running tests where I have a very fast producer
>>>>>>> and a
>>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>>> messages to a
>>>>>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>>>>>> to a
>>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>>> whacked and
>>>>>>> the enqueue graph in visualvm completely levels off. This is more
>>>>>>> or less
>>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>>> However, the
>>>>>>> entire broker stalls. I have several queues that are filled and
>>>>>>> emptied
>>>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>>>> flatline
>>>>>>> as well, even though flow control shouldn't apply to them. Thats
>>>>>>> why I
>>>>>>> was interested to find out if you'd discovered some kind of fresh
>>>>>>> angle.
>>>>>>> Regards,
>>>>>>> Maarten
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>
>>>>
>>>> Rob Davies
>>>> http://twitter.com/rajdavies
>>>> I work here: http://fusesource.com
>>>> My Blog: http://rajdavies.blogspot.com/
>>>> I'm writing this: http://www.manning.com/snyder/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625537.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
Sure, I'll try that now. In the mean time, could you perhaps tell me why (and
how) that would help?
Thanks in advance,
Maarten
rajdavies wrote:
>
> Can you try increasing the cache size for KahaDB - to 10000 ? - see
> http://activemq.apache.org/kahadb.html
>
> cheers,
>
> Rob
> On 17 Feb 2010, at 14:39, Maarten_D wrote:
>
>>
>> No problem:
>>
>> <beans
>> xmlns="http://www.springframework.org/schema/beans"
>> xmlns:amq="http://activemq.apache.org/schema/core"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>> http://activemq.apache.org/schema/core
>> http://activemq.apache.org/schema/core/activemq-core.xsd
>> http://mortbay.com/schemas/jetty/1.0
>> http://jetty.mortbay.org/jetty.xsd">
>>
>> <!-- Allows us to use system properties as variables in this
>> configuration
>> file -->
>> <bean
>> class
>> =
>> "org
>> .springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>> <property name="location" value="file:/var/amq/
>> broker.properties" />
>> </bean>
>>
>> <!--
>>
>> ******************************************************************************************************************
>> ** ActiveMQ broker
>>
>> ******************************************************************************************************************
>> -->
>> <broker id="broker" useJmx="true" brokerName="broker" start="true"
>> xmlns="http://activemq.apache.org/schema/core"
>> dataDirectory="/var/amq" advisorySupport="false"
>> persistenceAdapter="#store">
>>
>> <destinationPolicy>
>> <policyMap>
>> <policyEntries>
>> <policyEntry queue=">" memoryLimit="64 mb"
>> producerFlowControl="false" />
>> <policyEntry topic=">" memoryLimit="64 mb"
>> producerFlowControl="true" />
>> </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>> <managementContext>
>> <managementContext useMBeanServer="true"
>> jmxDomainName="org.apache.activemq"
>> createMBeanServer="true"
>> createConnector="false"
>> connectorPort="1100"
>> connectorPath="/jmxrmi"/>
>> </managementContext>
>>
>> <persistenceAdapter id="store">
>> <kahaDB enableJournalDiskSyncs="false"
>> journalMaxFileLength="32mb"
>> enableIndexWriteAsync="true"
>> directory="/var/amq/broker"
>> indexWriteBatchSize="1000" />
>> </persistenceAdapter>
>>
>> <systemUsage>
>> <systemUsage>
>> <memoryUsage>
>> <memoryUsage limit="512 mb" />
>> </memoryUsage>
>> </systemUsage>
>> </systemUsage>
>>
>> <transportConnectors>
>> <transportConnector name="cearchive" uri="tcp://0.0.0.0:61616" />
>> </transportConnectors>
>> </broker>
>>
>> <!-- Here we start an embedded webserver for the admin console -->
>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
>> <connectors>
>> <nioConnector port="8161"/>
>> </connectors>
>> <handlers>
>> <webAppContext contextPath="/admin"
>> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/>
>> </handlers>
>> </jetty>
>> </beans>
>>
>>
>>
>> rajdavies wrote:
>>>
>>> can you send your broker config ?
>>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>>
>>>>
>>>> The topics and queues are filled using a Spring JMSTemplate that has
>>>> it's own
>>>> connection factory, and dequeuing is done by message listeners that
>>>> all have
>>>> their own connection. So everything should have its own connection,
>>>> let
>>>> alone session.
>>>>
>>>> I'll do another run on debug and see what it turns up.
>>>>
>>>>
>>>> Adrian A wrote:
>>>>>
>>>>> you are running separate sessions for each of those dequeue/enqueue
>>>>> stats?
>>>>>
>>>>> in my flow control tests even when one particular session was hung
>>>>> other
>>>>> sessions to the same broker was fine, just when I overwhelmed
>>>>> broker and
>>>>> GC / disk checkpointing occurred that it got really bad.
>>>>>
>>>>> have you turned on debugging as that although verbose is a wealth
>>>>> of
>>>>> information!
>>>>>
>>>>>
>>>>>
>>>>> Maarten_D wrote:
>>>>>>
>>>>>> Hi Adrian, thanks for your response.
>>>>>>
>>>>>> I'm currently running tests where I have a very fast producer
>>>>>> and a
>>>>>> relatively slow consumer. The producer publishes persistent
>>>>>> messages to a
>>>>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>>>>> to a
>>>>>> difference of around 80,000 messages. The producer then gets
>>>>>> whacked and
>>>>>> the enqueue graph in visualvm completely levels off. This is more
>>>>>> or less
>>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>>> However, the
>>>>>> entire broker stalls. I have several queues that are filled and
>>>>>> emptied
>>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>>> flatline
>>>>>> as well, even though flow control shouldn't apply to them. Thats
>>>>>> why I
>>>>>> was interested to find out if you'd discovered some kind of fresh
>>>>>> angle.
>>>>>> Regards,
>>>>>> Maarten
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>
>>> Rob Davies
>>> http://twitter.com/rajdavies
>>> I work here: http://fusesource.com
>>> My Blog: http://rajdavies.blogspot.com/
>>> I'm writing this: http://www.manning.com/snyder/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27625224.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Rob Davies <ra...@gmail.com>.
Can you try increasing the cache size for KahaDB - to 10000 ? - see http://activemq.apache.org/kahadb.html
cheers,
Rob
On 17 Feb 2010, at 14:39, Maarten_D wrote:
>
> No problem:
>
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:amq="http://activemq.apache.org/schema/core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core.xsd
> http://mortbay.com/schemas/jetty/1.0
> http://jetty.mortbay.org/jetty.xsd">
>
> <!-- Allows us to use system properties as variables in this
> configuration
> file -->
> <bean
> class
> =
> "org
> .springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="location" value="file:/var/amq/
> broker.properties" />
> </bean>
>
> <!--
>
> ******************************************************************************************************************
> ** ActiveMQ broker
>
> ******************************************************************************************************************
> -->
> <broker id="broker" useJmx="true" brokerName="broker" start="true"
> xmlns="http://activemq.apache.org/schema/core"
> dataDirectory="/var/amq" advisorySupport="false"
> persistenceAdapter="#store">
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry queue=">" memoryLimit="64 mb"
> producerFlowControl="false" />
> <policyEntry topic=">" memoryLimit="64 mb"
> producerFlowControl="true" />
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <managementContext>
> <managementContext useMBeanServer="true"
> jmxDomainName="org.apache.activemq"
> createMBeanServer="true"
> createConnector="false"
> connectorPort="1100"
> connectorPath="/jmxrmi"/>
> </managementContext>
>
> <persistenceAdapter id="store">
> <kahaDB enableJournalDiskSyncs="false"
> journalMaxFileLength="32mb"
> enableIndexWriteAsync="true"
> directory="/var/amq/broker"
> indexWriteBatchSize="1000" />
> </persistenceAdapter>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="512 mb" />
> </memoryUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="cearchive" uri="tcp://0.0.0.0:61616" />
> </transportConnectors>
> </broker>
>
> <!-- Here we start an embedded webserver for the admin console -->
> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
> <connectors>
> <nioConnector port="8161"/>
> </connectors>
> <handlers>
> <webAppContext contextPath="/admin"
> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/>
> </handlers>
> </jetty>
> </beans>
>
>
>
> rajdavies wrote:
>>
>> can you send your broker config ?
>> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>>
>>>
>>> The topics and queues are filled using a Spring JMSTemplate that has
>>> it's own
>>> connection factory, and dequeuing is done by message listeners that
>>> all have
>>> their own connection. So everything should have its own connection,
>>> let
>>> alone session.
>>>
>>> I'll do another run on debug and see what it turns up.
>>>
>>>
>>> Adrian A wrote:
>>>>
>>>> you are running separate sessions for each of those dequeue/enqueue
>>>> stats?
>>>>
>>>> in my flow control tests even when one particular session was hung
>>>> other
>>>> sessions to the same broker was fine, just when I overwhelmed
>>>> broker and
>>>> GC / disk checkpointing occurred that it got really bad.
>>>>
>>>> have you turned on debugging as that although verbose is a wealth
>>>> of
>>>> information!
>>>>
>>>>
>>>>
>>>> Maarten_D wrote:
>>>>>
>>>>> Hi Adrian, thanks for your response.
>>>>>
>>>>> I'm currently running tests where I have a very fast producer
>>>>> and a
>>>>> relatively slow consumer. The producer publishes persistent
>>>>> messages to a
>>>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>>>> to a
>>>>> difference of around 80,000 messages. The producer then gets
>>>>> whacked and
>>>>> the enqueue graph in visualvm completely levels off. This is more
>>>>> or less
>>>>> expected, as I've turned on producerFlowControl for topics.
>>>>> However, the
>>>>> entire broker stalls. I have several queues that are filled and
>>>>> emptied
>>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>>> flatline
>>>>> as well, even though flow control shouldn't apply to them. Thats
>>>>> why I
>>>>> was interested to find out if you'd discovered some kind of fresh
>>>>> angle.
>>>>> Regards,
>>>>> Maarten
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
No problem:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
http://mortbay.com/schemas/jetty/1.0
http://jetty.mortbay.org/jetty.xsd">
<!-- Allows us to use system properties as variables in this configuration
file -->
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="file:/var/amq/broker.properties" />
</bean>
<!--
******************************************************************************************************************
** ActiveMQ broker
******************************************************************************************************************
-->
<broker id="broker" useJmx="true" brokerName="broker" start="true"
xmlns="http://activemq.apache.org/schema/core"
dataDirectory="/var/amq" advisorySupport="false"
persistenceAdapter="#store">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" memoryLimit="64 mb"
producerFlowControl="false" />
<policyEntry topic=">" memoryLimit="64 mb"
producerFlowControl="true" />
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext useMBeanServer="true"
jmxDomainName="org.apache.activemq"
createMBeanServer="true"
createConnector="false"
connectorPort="1100"
connectorPath="/jmxrmi"/>
</managementContext>
<persistenceAdapter id="store">
<kahaDB enableJournalDiskSyncs="false"
journalMaxFileLength="32mb"
enableIndexWriteAsync="true"
directory="/var/amq/broker"
indexWriteBatchSize="1000" />
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="512 mb" />
</memoryUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="cearchive" uri="tcp://0.0.0.0:61616" />
</transportConnectors>
</broker>
<!-- Here we start an embedded webserver for the admin console -->
<jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
<connectors>
<nioConnector port="8161"/>
</connectors>
<handlers>
<webAppContext contextPath="/admin"
resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/>
</handlers>
</jetty>
</beans>
rajdavies wrote:
>
> can you send your broker config ?
> On 17 Feb 2010, at 12:38, Maarten_D wrote:
>
>>
>> The topics and queues are filled using a Spring JMSTemplate that has
>> it's own
>> connection factory, and dequeuing is done by message listeners that
>> all have
>> their own connection. So everything should have its own connection,
>> let
>> alone session.
>>
>> I'll do another run on debug and see what it turns up.
>>
>>
>> Adrian A wrote:
>>>
>>> you are running separate sessions for each of those dequeue/enqueue
>>> stats?
>>>
>>> in my flow control tests even when one particular session was hung
>>> other
>>> sessions to the same broker was fine, just when I overwhelmed
>>> broker and
>>> GC / disk checkpointing occurred that it got really bad.
>>>
>>> have you turned on debugging as that although verbose is a wealth of
>>> information!
>>>
>>>
>>>
>>> Maarten_D wrote:
>>>>
>>>> Hi Adrian, thanks for your response.
>>>>
>>>> I'm currently running tests where I have a very fast producer and a
>>>> relatively slow consumer. The producer publishes persistent
>>>> messages to a
>>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>>> to a
>>>> difference of around 80,000 messages. The producer then gets
>>>> whacked and
>>>> the enqueue graph in visualvm completely levels off. This is more
>>>> or less
>>>> expected, as I've turned on producerFlowControl for topics.
>>>> However, the
>>>> entire broker stalls. I have several queues that are filled and
>>>> emptied
>>>> at the same time as the topic, and their dequeue/enqueue stats
>>>> flatline
>>>> as well, even though flow control shouldn't apply to them. Thats
>>>> why I
>>>> was interested to find out if you'd discovered some kind of fresh
>>>> angle.
>>>> Regards,
>>>> Maarten
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27624666.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Rob Davies <ra...@gmail.com>.
can you send your broker config ?
On 17 Feb 2010, at 12:38, Maarten_D wrote:
>
> The topics and queues are filled using a Spring JMSTemplate that has
> it's own
> connection factory, and dequeuing is done by message listeners that
> all have
> their own connection. So everything should have its own connection,
> let
> alone session.
>
> I'll do another run on debug and see what it turns up.
>
>
> Adrian A wrote:
>>
>> you are running separate sessions for each of those dequeue/enqueue
>> stats?
>>
>> in my flow control tests even when one particular session was hung
>> other
>> sessions to the same broker was fine, just when I overwhelmed
>> broker and
>> GC / disk checkpointing occurred that it got really bad.
>>
>> have you turned on debugging as that although verbose is a wealth of
>> information!
>>
>>
>>
>> Maarten_D wrote:
>>>
>>> Hi Adrian, thanks for your response.
>>>
>>> I'm currently running tests where I have a very fast producer and a
>>> relatively slow consumer. The producer publishes persistent
>>> messages to a
>>> topic, where the enqueue and dequeue count diverge fairly rapidly
>>> to a
>>> difference of around 80,000 messages. The producer then gets
>>> whacked and
>>> the enqueue graph in visualvm completely levels off. This is more
>>> or less
>>> expected, as I've turned on producerFlowControl for topics.
>>> However, the
>>> entire broker stalls. I have several queues that are filled and
>>> emptied
>>> at the same time as the topic, and their dequeue/enqueue stats
>>> flatline
>>> as well, even though flow control shouldn't apply to them. Thats
>>> why I
>>> was interested to find out if you'd discovered some kind of fresh
>>> angle.
>>> Regards,
>>> Maarten
>>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
The topics and queues are filled using a Spring JMSTemplate that has it's own
connection factory, and dequeuing is done by message listeners that all have
their own connection. So everything should have its own connection, let
alone session.
I'll do another run on debug and see what it turns up.
Adrian A wrote:
>
> you are running separate sessions for each of those dequeue/enqueue stats?
>
> in my flow control tests even when one particular session was hung other
> sessions to the same broker was fine, just when I overwhelmed broker and
> GC / disk checkpointing occurred that it got really bad.
>
> have you turned on debugging as that although verbose is a wealth of
> information!
>
>
>
> Maarten_D wrote:
>>
>> Hi Adrian, thanks for your response.
>>
>> I'm currently running tests where I have a very fast producer and a
>> relatively slow consumer. The producer publishes persistent messages to a
>> topic, where the enqueue and dequeue count diverge fairly rapidly to a
>> difference of around 80,000 messages. The producer then gets whacked and
>> the enqueue graph in visualvm completely levels off. This is more or less
>> expected, as I've turned on producerFlowControl for topics. However, the
>> entire broker stalls. I have several queues that are filled and emptied
>> at the same time as the topic, and their dequeue/enqueue stats flatline
>> as well, even though flow control shouldn't apply to them. Thats why I
>> was interested to find out if you'd discovered some kind of fresh angle.
>> Regards,
>> Maarten
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27623064.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Adrian A <ad...@acta.co.uk>.
you are running separate sessions for each of those dequeue/enqueue stats?
in my flow control tests even when one particular session was hung other
sessions to the same broker was fine, just when I overwhelmed broker and GC
/ disk checkpointing occurred that it got really bad.
have you turned on debugging as that although verbose is a wealth of
information!
Maarten_D wrote:
>
> Hi Adrian, thanks for your response.
>
> I'm currently running tests where I have a very fast producer and a
> relatively slow consumer. The producer publishes persistent messages to a
> topic, where the enqueue and dequeue count diverge fairly rapidly to a
> difference of around 80,000 messages. The producer then gets whacked and
> the enqueue graph in visualvm completely levels off. This is more or less
> expected, as I've turned on producerFlowControl for topics. However, the
> entire broker stalls. I have several queues that are filled and emptied at
> the same time as the topic, and their dequeue/enqueue stats flatline as
> well, even though flow control shouldn't apply to them. Thats why I was
> interested to find out if you'd discovered some kind of fresh angle.
> Regards,
> Maarten
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27612368.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Maarten_D <ma...@gmail.com>.
Hi Adrian, thanks for your response.
I'm currently running tests where I have a very fast producer and a
relatively slow consumer. The producer publishes persistent messages to a
topic, where the enqueue and dequeue count diverge fairly rapidly to a
difference of around 80,000 messages. The producer then gets whacked and the
enqueue graph in visualvm completely levels off. This is more or less
expected, as I've turned on producerFlowControl for topics. However, the
entire broker stalls. I have several queues that are filled and emptied at
the same time as the topic, and their dequeue/enqueue stats flatline as
well, even though flow control shouldn't apply to them. Thats why I was
interested to find out if you'd discovered some kind of fresh angle.
Regards,
Maarten
Adrian A wrote:
>
> Hi Maarten
>
> Kinda.
>
> MapMessages are significantly more expensive but no real surprise there.
>
> I upgraded the machine and did significant optimising on the GC, paying
> particular attention to parallel / conc mark sweep
>
> eg
> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing
> -XX:CMSIncrementalDutyCycleMin=0
> or
> -XX:+UseParallelGC"
> or
> -XX:-UseParallelOldGC
>
> I couldn't find any good info on what GC settings people have used for
> their platform.... but working with known test cases and measuring over
> multiple 10,000's tests helped to determine optimum settings on mine.
>
> what are you seeing?
>
> thanks
>
> Adrian
>
>
>
> Maarten_D wrote:
>>
>> Hi Adrian,
>> Have you made any progress on this issue? I believe I'm running into the
>> same things occasionally.
>> Regards,
>> Maarten
>>
>>
>> Adrian A wrote:
>>>
>>> Setting them to MapMessages (what I am using) makes it even worse:
>>>
>>> //TextMessage message = session.createTextMessage(createMessageText(i));
>>> MapMessage message = session.createMapMessage();
>>> message.setJMSType("blahasdadasdsd");
>>> message.setLong("test1",2222);
>>> message.setLong("test2",2222);
>>> message.setLong("test3",2222);
>>> message.setLong("test4",2222);
>>> message.setLong("test5",2222);
>>>
>>> by 10 to 20 times worse.
>>>
>>>
>>> Adrian A wrote:
>>>>
>>>> long start = System.currentTimeMillis();
>>>> producer.send(message);
>>>> long stop = System.currentTimeMillis();
>>>>
>>>
>>>
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27612143.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Slow sending of messages
Posted by Adrian A <ad...@acta.co.uk>.
Hi Maarten
Kinda.
MapMessages are significantly more expensive but no real surprise there.
I upgraded the machine and did significant optimising on the GC, paying
particular attention to parallel / conc mark sweep
eg
-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing
-XX:CMSIncrementalDutyCycleMin=0
or
-XX:+UseParallelGC"
or
-XX:-UseParallelOldGC
I couldn't find any good info on what GC settings people have used for their
platform.... but working with known test cases and measuring over multiple
10,000's tests helped to determine optimum settings on mine.
what are you seeing?
thanks
Adrian
Maarten_D wrote:
>
> Hi Adrian,
> Have you made any progress on this issue? I believe I'm running into the
> same things occasionally.
> Regards,
> Maarten
>
>
> Adrian A wrote:
>>
>> Setting them to MapMessages (what I am using) makes it even worse:
>>
>> //TextMessage message = session.createTextMessage(createMessageText(i));
>> MapMessage message = session.createMapMessage();
>> message.setJMSType("blahasdadasdsd");
>> message.setLong("test1",2222);
>> message.setLong("test2",2222);
>> message.setLong("test3",2222);
>> message.setLong("test4",2222);
>> message.setLong("test5",2222);
>>
>> by 10 to 20 times worse.
>>
>>
>> Adrian A wrote:
>>>
>>> long start = System.currentTimeMillis();
>>> producer.send(message);
>>> long stop = System.currentTimeMillis();
>>>
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Slow-sending-of-messages-tp26849964p27611873.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.