You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Vadim Chekan (JIRA)" <ji...@apache.org> on 2008/09/17 09:15:52 UTC
[jira] Created: (AMQ-1940) Negative queue size (reproducible)
Negative queue size (reproducible)
----------------------------------
Key: AMQ-1940
URL: https://issues.apache.org/activemq/browse/AMQ-1940
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.2.0
Environment: Found on Windows but reproduced under Linux
Reporter: Vadim Chekan
Priority: Critical
When you "purge" queue from web admin console, it zeroes queue message
counter. But if you have an active consumer at that time which
pre-fetched messages than your consumer will keep sending ack as it
process messages from its buffer. ActiveMQ will keep decrement counter
upon receiving each ack. So when consumer is done queue will show
MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1940) Negative queue size (reproducible)
Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Snyder updated AMQ-1940:
------------------------------
Attachment: QueuePurgeTest.java.diff.txt
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Attachments: Main.java, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (AMQ-1940) Negative queue size (reproducible)
Posted by "Brian Moran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Moran reopened AMQ-1940:
------------------------------
Regression: [Regression]
I can reproduce this in 5.2.0.
I am going to attach a screenshot.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Loc Truong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50856#action_50856 ]
Loc Truong commented on AMQ-1940:
---------------------------------
I can reproduct this in 5.2.0. This happens to me because there are too many receivers, each of them try to recevie 500 messages but the total messages in queue is smaller, hence negative number.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Vadim Chekan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=45778#action_45778 ]
Vadim Chekan commented on AMQ-1940:
-----------------------------------
Perhaps it can be made more reproducible if purge is run in a thread parallel to the consumer. I'll try to rewrite it later tonight.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Attachments: Main.java, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AMQ-1940) Negative queue size (reproducible)
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies reassigned AMQ-1940:
-------------------------------
Assignee: Rob Davies
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Attachments: Main.java
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Ivan S. Dubrov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51794#action_51794 ]
Ivan S. Dubrov commented on AMQ-1940:
-------------------------------------
I had the same issue today.
We are running two AMQ brokers in network of brokers, each one talking to the other one (the duplex is set to false). Under a high load after few hours I've noted that one of our queues started growing on both nodes, several thousand of pending messages in about 20 minutes (and the cursor memory usage was quite large). After I accidentally stopped the second node, the network bridge stopped on the first node and the queue size on the first node immediately became "-8".
So in my case this probably somehow related to the instability of network of brokers implementation, because if I disable the network connector the system runs stable for several days under the load.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1940) Negative queue size (reproducible)
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1940.
-----------------------------
Resolution: Fixed
Fix Version/s: (was: 5.2.0)
5.3.0
Fixed for the 5.3
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.3.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1940) Negative queue size (reproducible)
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1940.
-----------------------------
Fix Version/s: 5.3.0
Resolution: Fixed
Fixed by SVN revision 697921
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.3.0
>
> Attachments: Main.java, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Yuri Ushakov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50862#action_50862 ]
Yuri Ushakov commented on AMQ-1940:
-----------------------------------
We have the same problem with 5.2.0. A queue, single consumer, single producer. We have -1 pending now, 1616 sent, 1617 received.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1940) Negative queue size (reproducible)
Posted by "Brian Moran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Moran updated AMQ-1940:
-----------------------------
Attachment: Picture 6.png
Screenshot of negative messages, reproducible in 5.2.0 after purging the message queue, STOMP clients.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Loc Truong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50991#action_50991 ]
Loc Truong commented on AMQ-1940:
---------------------------------
I can reproduce this on 5.2.0 too. I have 7 consumers getting 150 messages each. The negative number I receive is -123 at lowest.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Assignee: Rob Davies
> Priority: Critical
> Fix For: 5.2.0
>
> Attachments: Main.java, Picture 6.png, QueuePurgeTest.java.diff.txt
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1940) Negative queue size (reproducible)
Posted by "Vadim Chekan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vadim Chekan updated AMQ-1940:
------------------------------
Attachment: Main.java
Attached program reproduces a queue with -100 messages.
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Priority: Critical
> Attachments: Main.java
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1940) Negative queue size (reproducible)
Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=45756#action_45756 ]
Bruce Snyder commented on AMQ-1940:
-----------------------------------
I've created the following test case out of the attached {{Main.java}} class:
{code}
package org.apache.activemq.broker.region;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.management.MBeanServerInvocationHandler;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import junit.framework.TestCase;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.broker.BrokerService;
import org.apache.activemq.broker.jmx.QueueViewMBean;
public class QueuePurgeTest extends TestCase {
BrokerService broker;
ConnectionFactory factory;
Connection connection;
Session session;
Queue queue;
MessageConsumer consumer;
protected void setUp() throws Exception {
broker = new BrokerService();
broker.setUseJmx(true);
broker.setPersistent(false);
broker.addConnector("tcp://localhost:0");
broker.start();
factory = new ActiveMQConnectionFactory("vm://localhost");
connection = factory.createConnection();
connection.start();
}
protected void tearDown() throws Exception {
consumer.close();
session.close();
connection.stop();
connection.close();
broker.stop();
}
public void testPurgeQueueWithActiveConsumer() throws Exception {
createProducerAndSendMessages();
QueueViewMBean proxy = getProxyToQueueViewMBean();
createConsumer();
proxy.purge();
assertEquals("Queue size is not zero, it's " + proxy.getQueueSize(), 0, proxy.getQueueSize());
}
private QueueViewMBean getProxyToQueueViewMBean()
throws MalformedObjectNameException, JMSException {
ObjectName queueViewMBeanName = new ObjectName("org.apache.activemq" + ":Type=Queue,Destination=" +
queue.getQueueName() + ",BrokerName=localhost");
QueueViewMBean proxy = (QueueViewMBean)MBeanServerInvocationHandler.newProxyInstance(
broker.getManagementContext().getMBeanServer(),
queueViewMBeanName, QueueViewMBean.class, true);
return proxy;
}
private void createProducerAndSendMessages() throws Exception {
session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
queue = session.createQueue("test1");
MessageProducer producer = session.createProducer(queue);
for(int i=0; i<10000; i++) {
TextMessage message = session.createTextMessage("message "+i);
producer.send(message);
}
producer.close();
}
private void createConsumer() throws Exception {
consumer = session.createConsumer(queue);
// wait for buffer fill out
Thread.sleep(5*1000);
for(int i = 0; i < 100; ++i) {
Message message = consumer.receive();
message.acknowledge();
}
}
}
{code}
What I'm finding is that failure is intermittent, but I am able to see it once in a while:
{panel}
junit.framework.AssertionFailedError: Queue size is not zero expected:<0> but was:<-60>
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:282)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:136)
at org.apache.activemq.broker.region.QueuePurgeTest.testPurgeQueueWithActiveConsumer(QueuePurgeTest.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
{panel}
> Negative queue size (reproducible)
> ----------------------------------
>
> Key: AMQ-1940
> URL: https://issues.apache.org/activemq/browse/AMQ-1940
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Environment: Found on Windows but reproduced under Linux
> Reporter: Vadim Chekan
> Priority: Critical
> Attachments: Main.java
>
>
> When you "purge" queue from web admin console, it zeroes queue message
> counter. But if you have an active consumer at that time which
> pre-fetched messages than your consumer will keep sending ack as it
> process messages from its buffer. ActiveMQ will keep decrement counter
> upon receiving each ack. So when consumer is done queue will show
> MINUS<consumer buffer size>.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.