You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Jim Plush (JIRA)" <ji...@apache.org> on 2010/11/13 00:37:24 UTC
[jira] Created: (AMQ-3034) Consumers hang when flow control is
enabled - unable to proceed?
Consumers hang when flow control is enabled - unable to proceed?
----------------------------------------------------------------
Key: AMQ-3034
URL: https://issues.apache.org/activemq/browse/AMQ-3034
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.5.0
Environment: ubuntu linux 9.10
Reporter: Jim Plush
Priority: Critical
I was seeing odd behavior in production and created a test case around the follow issue....
I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
Expected Result:
Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
Producer Code: http://pastebin.com/stAN5UJv
Consumer Code: http://pastebin.com/dWuWWATr
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (AMQ-3034) Consumers hang when flow
control is enabled - unable to proceed?
Posted by "Jim Plush (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=63329#action_63329 ]
Jim Plush edited comment on AMQ-3034 at 11/12/10 6:59 PM:
----------------------------------------------------------
more notes:
When it's in the blocking state and I stop all producers and all consumers so nothing is hitting the broker then I go to the web admin panel and hit "delete" to remove the queue I get hit with some errors in my activemq log file in the data dir
http://pastebin.com/q7mxzGzK
I cannot purge or delete this queue at this time, the browser just hangs. http://cache07:8161/admin/queues.jsp (internal host)
When I now run bin/activemq stop localhost I'm seeing this in my log file even though there are no producers or consumers attached to it any longer:
2010-11-12 15:56:40,265 | INFO | The connection to '/10.128.129.20:33830' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ ShutdownHook
2010-11-12 15:56:40,827 | INFO | The connection to '/10.128.129.20:33830' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | Thread-22
I'm in a state where I have to kill -9 the process to get the server back.
was (Author: jiminoc):
more notes:
When it's in the blocking state and I stop all producers and all consumers so nothing is hitting the broker then I go to the web admin panel and hit "delete" to remove the queue I get hit with some errors in my activemq log file in the data dir
http://pastebin.com/q7mxzGzK
I cannot purge or delete this queue at this time, the browser just hangs. http://cache07:8161/admin/queues.jsp (internal host)
> Consumers hang when flow control is enabled - unable to proceed?
> ----------------------------------------------------------------
>
> Key: AMQ-3034
> URL: https://issues.apache.org/activemq/browse/AMQ-3034
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.5.0
> Environment: ubuntu linux 9.10
> Reporter: Jim Plush
> Priority: Critical
>
> I was seeing odd behavior in production and created a test case around the follow issue....
> I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
> about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
> 2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
> which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
> Expected Result:
> Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
> Producer Code: http://pastebin.com/stAN5UJv
> Consumer Code: http://pastebin.com/dWuWWATr
> Version: apache-activemq-5.5-SNAPSHOT
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-3034) Consumers hang when flow control is
enabled - unable to proceed?
Posted by "Jim Plush (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=63329#action_63329 ]
Jim Plush commented on AMQ-3034:
--------------------------------
more notes:
When it's in the blocking state and I stop all producers and all consumers so nothing is hitting the broker then I go to the web admin panel and hit "delete" to remove the queue I get hit with some errors in my activemq log file in the data dir
http://pastebin.com/q7mxzGzK
I cannot purge or delete this queue at this time, the browser just hangs. http://cache07:8161/admin/queues.jsp (internal host)
> Consumers hang when flow control is enabled - unable to proceed?
> ----------------------------------------------------------------
>
> Key: AMQ-3034
> URL: https://issues.apache.org/activemq/browse/AMQ-3034
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.5.0
> Environment: ubuntu linux 9.10
> Reporter: Jim Plush
> Priority: Critical
>
> I was seeing odd behavior in production and created a test case around the follow issue....
> I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
> about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
> 2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
> which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
> Expected Result:
> Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
> Producer Code: http://pastebin.com/stAN5UJv
> Consumer Code: http://pastebin.com/dWuWWATr
> Version: apache-activemq-5.5-SNAPSHOT
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-3034) Consumers hang when flow control is
enabled - unable to proceed?
Posted by "Jim Plush (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Plush updated AMQ-3034:
---------------------------
Description:
I was seeing odd behavior in production and created a test case around the follow issue....
I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
Expected Result:
Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
Producer Code: http://pastebin.com/stAN5UJv
Consumer Code: http://pastebin.com/dWuWWATr
Version: apache-activemq-5.5-SNAPSHOT
was:
I was seeing odd behavior in production and created a test case around the follow issue....
I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
Expected Result:
Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
Producer Code: http://pastebin.com/stAN5UJv
Consumer Code: http://pastebin.com/dWuWWATr
> Consumers hang when flow control is enabled - unable to proceed?
> ----------------------------------------------------------------
>
> Key: AMQ-3034
> URL: https://issues.apache.org/activemq/browse/AMQ-3034
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.5.0
> Environment: ubuntu linux 9.10
> Reporter: Jim Plush
> Priority: Critical
>
> I was seeing odd behavior in production and created a test case around the follow issue....
> I have a producer that will attempt to enqueue 1 million messages in a while loop. I set the activemq.xml systemUsage settings to the following http://pastebin.com/hJXnPHKK very low to reproduce the problem.
> about 1700 messages in I hit the memory and tempUsage limits for NON_PERSISTENT messages. My activemq log file now says:
> 2010-11-12 15:25:28,777 | INFO | Usage Manager Temp Store is Full (99% of 104857600). Stopping producer (ID:grv-crawl04-35650-1289603606125-0:0:1:1) to prevent flooding queue://benchmarks2. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 722s) | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///10.128.129.20:38430
> which is expected as I just hit flow control. On another machine I fire up a consumer to try and dequeue messages while the producer is blocking. It's now in a state where I can't enqueue OR dequeue as the consumer times out on the .receive(10000) command. The only thing I can do at this point is restart the server which loses all the NON_PERSISTENT messages.
> Expected Result:
> Producer would hang when resources are maxed out, fire up a Consumer on another machine and it starts to dequeue and free up space, the producer slowly starts to enqueue messages again.
> Producer Code: http://pastebin.com/stAN5UJv
> Consumer Code: http://pastebin.com/dWuWWATr
> Version: apache-activemq-5.5-SNAPSHOT
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.