You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Hiram Chirino (JIRA)" <ji...@apache.org> on 2008/07/22 19:35:00 UTC
[jira] Created: (AMQ-1866) Using small queue prefetch sizes cause
all consumers to run as fast as the slowest consumer.
Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.
--------------------------------------------------------------------------------------------
Key: AMQ-1866
URL: https://issues.apache.org/activemq/browse/AMQ-1866
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.1.0
Reporter: Hiram Chirino
Assignee: Hiram Chirino
If you have a JMS producer sending messages to multiple consumers and you have the cacheLevel set to 3 or higher, eventually all consumers will run as slow as the slowest consumer.
In the attached projects the SA sends messages to a MockOptimizer (JMS consumer) that has two threads. One sleeps for one second and the other sleeps for ten. Initially the thread sleeping for 1 second will consume most of the messages, but towards the end it will start to consume messages every ten seconds as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Using small queue prefetch sizes cause
all consumers to run as fast as the slowest consumer.
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44438#action_44438 ]
Hiram Chirino commented on AMQ-1866:
------------------------------------
test case AMQ1866.java has been added to trunk. Currently failing it show a test case the works for the default prefetch but which fails for small prefetches.
> Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.
> --------------------------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Hiram Chirino
>
> If you have a JMS producer sending messages to multiple consumers and you have the cacheLevel set to 3 or higher, eventually all consumers will run as slow as the slowest consumer.
> In the attached projects the SA sends messages to a MockOptimizer (JMS consumer) that has two threads. One sleeps for one second and the other sleeps for ten. Initially the thread sleeping for 1 second will consume most of the messages, but towards the end it will start to consume messages every ten seconds as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "David Sitsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44705#action_44705 ]
David Sitsky commented on AMQ-1866:
-----------------------------------
Just curious does this bug occur if lazy dispatch is enabled? Really the best way of solving this is to not deliver messages to consumers that aren't waiting for a message. I would hope if lazy-dispatch is enabled, this doesn't happen..
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hiram Chirino updated AMQ-1866:
-------------------------------
Summary: Muliple consumers on a queue and only run as fast as the slowest consumer (was: Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.)
Description: Muliple consumers on a queue and only run as fast as the slowest consumer (was: If you have a JMS producer sending messages to multiple consumers and you have the cacheLevel set to 3 or higher, eventually all consumers will run as slow as the slowest consumer.
In the attached projects the SA sends messages to a MockOptimizer (JMS consumer) that has two threads. One sleeps for one second and the other sleeps for ten. Initially the thread sleeping for 1 second will consume most of the messages, but towards the end it will start to consume messages every ten seconds as well.)
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Hiram Chirino
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=45107#action_45107 ]
Gary Tully commented on AMQ-1866:
---------------------------------
if ye get a moment please cast your eyes over revision 687677 - fix for AMQ-1902
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Using small queue prefetch sizes cause
all consumers to run as fast as the slowest consumer.
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44439#action_44439 ]
Hiram Chirino commented on AMQ-1866:
------------------------------------
Seems it's not limited to small prefetch sizes. Test case has been updated and shows that the slow down will happen even with the default prefetch size.
> Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.
> --------------------------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Hiram Chirino
>
> If you have a JMS producer sending messages to multiple consumers and you have the cacheLevel set to 3 or higher, eventually all consumers will run as slow as the slowest consumer.
> In the attached projects the SA sends messages to a MockOptimizer (JMS consumer) that has two threads. One sleeps for one second and the other sleeps for ten. Initially the thread sleeping for 1 second will consume most of the messages, but towards the end it will start to consume messages every ten seconds as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hiram Chirino updated AMQ-1866:
-------------------------------
Attachment: AMQ1866.patch
Attaching patch which fixes the test case. Submitting for review to make sure there are no other performance/memory implications.
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Hiram Chirino
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44550#action_44550 ]
Hiram Chirino commented on AMQ-1866:
------------------------------------
The issue revolves around how ActiveMQ 5.x pages and dispatches messages to consumers. Every queue keeps list of paged in messages which defaults to 100 max entires. It round robins dispatching messages between consumers, even the slow consumers. Once the slow consumer gets 100 dispatched messages which it has not yet acked due to it being slow, then no further messages are paged in since the page in list is full.
I am evaluating different ways to implement this but, any change to this code could have repercussions which need to get evaluated.
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Hiram Chirino
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies reopened AMQ-1866:
-----------------------------
The fix doesn't work if the consumer is slightly slow ...
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1866.
-----------------------------
Fix Version/s: 5.2.0
Resolution: Fixed
Using pendingQueueSize to determine least loaded subscription seems the best way to go
Fixed by SVN revision 680161
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44956#action_44956 ]
Rob Davies commented on AMQ-1866:
---------------------------------
Hi David - we intend change the dispatch model so that it works this way - if you get a chance - would be very happy if you could give it a run ?
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "David Sitsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44977#action_44977 ]
David Sitsky commented on AMQ-1866:
-----------------------------------
No worries - I'll try and set aside some time next week to check it out.
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1866.
-----------------------------
Resolution: Fixed
Fixed by SVN revision 685806
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Fix For: 5.2.0
>
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AMQ-1866) Muliple consumers on a queue and only
run as fast as the slowest consumer
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies reassigned AMQ-1866:
-------------------------------
Assignee: Rob Davies (was: Hiram Chirino)
> Muliple consumers on a queue and only run as fast as the slowest consumer
> -------------------------------------------------------------------------
>
> Key: AMQ-1866
> URL: https://issues.apache.org/activemq/browse/AMQ-1866
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.1.0
> Reporter: Hiram Chirino
> Assignee: Rob Davies
> Attachments: AMQ1866.patch
>
>
> Muliple consumers on a queue and only run as fast as the slowest consumer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.