You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Jürgen Schumacher (JIRA)" <ji...@apache.org> on 2008/01/21 10:39:21 UTC
[jira] Created: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
High message frequency causes ActiveMQ to freeze
------------------------------------------------
Key: AMQ-1557
URL: https://issues.apache.org/activemq/browse/AMQ-1557
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.0.0, 5.1.0
Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
Reporter: Jürgen Schumacher
Attachments: stresstest.zip
ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
- Host 1: Multiple Producers send messages very fast to queue1
- Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
- Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
Using transactions or not does not seem to make a difference.
See this thread in user mailing list for more details:
http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Grant Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=42007#action_42007 ]
Grant Smith commented on AMQ-1557:
----------------------------------
For me I'm getting :
ERROR RecoveryListenerAdapter - Message id ID:gsdev2.marathon-man.com-58448-1207350845238-0:0:1:1:2122 could not be recovered from the data store - already dispatched
ERROR RecoveryListenerAdapter - Message id ID:gsdev2.marathon-man.com-58448-1207350845238-0:0:1:1:2123 could not be recovered from the data store - already dispatched
multiple times under high bandwidth conditions. It appears to be a manifestation of
https://issues.apache.org/activemq/browse/AMQ-1544 which appears to NOT be resolved ?
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Jürgen Schumacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jürgen Schumacher updated AMQ-1557:
-----------------------------------
Attachment: stresstest.zip
re-attached with "Grant license..."
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Filip Hanik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=42494#action_42494 ]
Filip Hanik commented on AMQ-1557:
----------------------------------
If I reconfigure the dead letter strategy
<deadLetterStrategy>
<sharedDeadLetterStrategy processExpired="false" processNonPersistent="false" />
</deadLetterStrategy>
and turn off the flooding of the logs that it causes
log4j.logger.org.apache.activemq.broker.region.RegionBroker=ERROR
The system seems to stay up, in this case, my client (consumer) OOMs, and during the time it OOMS, the producer freezes up again,
it seems that the producer is synchronized on a connected consumer, and there is no asynchronicity between the two
Filip
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies reassigned AMQ-1557:
-------------------------------
Assignee: Rob Davies
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Marco Buss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=41100#action_41100 ]
Marco Buss commented on AMQ-1557:
---------------------------------
I can see the same with topics async send and producer flow control. The persistence store was derby and mysql.
I used 3 Topics with 9 diferent subscribers for each topic. 5 of this subscribers were slow (they pause for 60 seconds after every message). All looks fine as long as the topic reaches 1000 Messages. Then the cpu amount used increse to 90% and no message can be delivered, but the Consumers already get the Messages.
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Jürgen Schumacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jürgen Schumacher updated AMQ-1557:
-----------------------------------
Attachment: (was: stresstest.zip)
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Grant Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=42006#action_42006 ]
Grant Smith commented on AMQ-1557:
----------------------------------
This still occurs with today's SNAPSHOT. Any updates on getting this fixed ?
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1557.
-----------------------------
Fix Version/s: 5.2.0
Resolution: Fixed
Resolved in 5.2
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Filip Hanik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=42493#action_42493 ]
Filip Hanik commented on AMQ-1557:
----------------------------------
I'm able to consistently reproduce the error with 5.0.0 and 5.1.0
I haven't been able to try it out trunk, since it doesn't build when I check it out.
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid3448.hprof ...
Heap dump file created [212044211 bytes in 14.203 secs]
Exception in thread "ActiveMQ Transport: tcp:///127.0.0.1:1206" java.lang.OutOfMemoryError: Java heap space
at org.apache.activemq.openwire.v3.BaseDataStreamMarshaller.tightUnmarshalByteSequence(BaseDataStreamMarshaller.
java:441)
at org.apache.activemq.openwire.v3.MessageMarshaller.tightUnmarshal(MessageMarshaller.java:68)
at org.apache.activemq.openwire.v3.ActiveMQMessageMarshaller.tightUnmarshal(ActiveMQMessageMarshaller.java:67)
at org.apache.activemq.openwire.v3.ActiveMQTextMessageMarshaller.tightUnmarshal(ActiveMQTextMessageMarshaller.ja
va:67)
at org.apache.activemq.openwire.OpenWireFormat.doUnmarshal(OpenWireFormat.java:361)
at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:192)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:184)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:172)
at java.lang.Thread.run(Thread.java:595)
Here is the info
1. Configuration file
http://people.apache.org/~fhanik/activemq.xml
2. Modified example code
http://people.apache.org/~fhanik/example.zip
Steps to reproduce
1. Apply the activemq.xml file
2. start the activemq server
3. start the consumer 'ant consumer' in the example directory (from example.zip)
4. start the producer 'ant producer' in the example directory (from example.zip)
Just sit and watch, eventually it starts slowing down, and finally it throws an OutOfMemoryError
As long as the consumer is connected, it's not skipping or discarding any messages.
I was expecting the consumer to receive only the latst, but as long as the consumer is connected, not a single message gets discarded, or at least it appears that way, until the server runs out of java heap.
Filip
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1557) High message frequency causes ActiveMQ
to freeze
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=42405#action_42405 ]
Rob Davies commented on AMQ-1557:
---------------------------------
Before we even schedule this issue - the attachment needs to be Granted to the ASF (tick the check box) and all files should be Apache licenced - could you delete and reapply?
> High message frequency causes ActiveMQ to freeze
> ------------------------------------------------
>
> Key: AMQ-1557
> URL: https://issues.apache.org/activemq/browse/AMQ-1557
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.0.0, 5.1.0
> Environment: observed on Windows XP Professional SP2, Sun JDK 1.5.01_11, but probably affects all platforms
> Reporter: Jürgen Schumacher
> Assignee: Rob Davies
> Attachments: stresstest.zip
>
>
> ActiveMQ freezes in a test setup with very high message frequency of persistent messages and multiple producers and consumers running on multiple machines:
> - Host 1: Multiple Producers send messages very fast to queue1
> - Host 2: Multiple Processors consume message from queue1 and send a new message for each consumed message to queue2, also very fast.
> - Host 3: ActiveMQ server and 1 consumer reads messages from queue2.
> With producerFlowControl enabled, the system freezes very soon (a bit more than 1000 messages).
> With producerFlowControl disabled, it seems to run longer at first, but it still freezes later.
> Using transactions or not does not seem to make a difference.
> See this thread in user mailing list for more details:
> http://www.nabble.com/High-message-frequency-causes-ActiveMQ-to-freeze-to14919292s2354.html
> See attachment for code of test cases and the batch files to start them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.