All messages in queue repeatedly vanish, but reappear after restart

[mcorgan <mc...@hotpads.com>]

I have been publishing 100,000 simple messages into a queue, and then having
~5 consumers pull them off the queue as fast as possible.  I haven't been
able to figure out exactly what triggers it, but usually the all the
messages in the queue will disappear.  The web console indicates that they
are in the "pending" state, but if I click the queue name in the web
console, there are no messages.  Restarting ActiveMQ brings back the
messages, but the pending message count is zero.  As the recovered messages
are consumed, the pending message count goes negative.

Does anyone know why these messages vanish?  There can be like 80,000 of
them disappear all at once.

I've noticed the problem occurs more frequently if I lower the receive
timeout, increase the number of consumers, or start/close a new session for
each message.  So, it's basically failing under load.

I'm basically trying to do round-robin distribution of document ids to job
processing servers.  It's for real estate listings on http://hotpads.com.  I
can't find anyone else with exactly this same problem, but it's occuring so
frequently that I must be doing something abnormal.  See anything strange in
my consumer code below?

Thanks,
Matt
mcorgan@hotpads.com



public class MessagingTest{	

	public static void main(String[] args)throws Exception{
		testConsumeSomeMessages();
	}
	

	public static final String TEST_PORT = "tcp://192.168.1.10:61616";
	public static final String TEST_QUEUE = "test";


	public static void testConsumeSomeMessages() throws Exception {
		ConnectionFactory connectionFactory = new
ActiveMQConnectionFactory(TEST_PORT);
		if (connectionFactory instanceof ActiveMQConnectionFactory) {
			ActiveMQPrefetchPolicy po = new ActiveMQPrefetchPolicy();
			po.setAll(1);
			((ActiveMQConnectionFactory) connectionFactory).setPrefetchPolicy(po);
		}
		Connection connection = connectionFactory.createConnection();
		connection.start();
		Session session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
		Destination destination = session.createQueue(TEST_QUEUE);
		MessageConsumer consumer = session.createConsumer(destination);
		TextMessage message = null;
		while (true) {
			message = (TextMessage) consumer.receive(5000);
			if (message == null){ break; }
			message.acknowledge();
		}
		consumer.close();
		session.close();
		connection.close();
	}
}

-- 
View this message in context: http://www.nabble.com/All-messages-in-queue-repeatedly-vanish%2C-but-reappear-after-restart-tp18406448p18406448.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.