You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Dave cawthorn <da...@empired.com> on 2006/07/04 12:25:09 UTC
AMQ 4.0.1 Memory Leak/Hang
Hi,
I have a simple test case that is failing at the moment. It goes like this
1 create a connection/session/producer
2 create a connection/session/consumer
3 create a message and send
4 readnext message on the consumer
5 close every thing
6 loop from 1 again
I've tried to configure amq to use persistence so I wont run into memory
problems (and can support failover with a replicated message store) but when
i run the test it just consumes memory (amq jvm) till it hangs the client
when trying to create a session.
here is my config - any ideas?
<!-- START SNIPPET: xbean -->
<beans xmlns="http://activemq.org/config/1.0">
<broker persistent="true" useJmx="false">
<memoryManager>
<usageManager id="memory-manager" limit="100000000"/>
</memoryManager>
<managementContext>
<managementContext connectorPort="1199"
jmxDomainName="org.apache.activemq"/>
</managementContext>
<!-- In ActiveMQ 4, you can setup destination policies -->
<destinationPolicy>
<policyMap><policyEntries>
<policyEntry topic=">" memoryLimit="50000000"/>
<policyEntry queue=">" memoryLimit="50000000"/>
</policyEntries></policyMap>
</destinationPolicy>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#sequoia-ds" />
</persistenceAdapter>
<transportConnectors>
<transportConnector name="default" uri="tcp://localhost:61616"
discoveryUri="multicast://default"/>
<transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors>
<networkConnectors>
<!-- by default just auto discover the other brokers
<networkConnector name="default" uri="multicast://default"/> -->
<!--
<networkConnector name="host1 and host2"
uri="static://(tcp://host1:61616,tcp://host2:61616)" failover="true"/>
-->
</networkConnectors>
</broker>
<!-- This xbean configuration file supports all the standard spring xml
configuration options -->
<!-- Postgres DataSource Sample Setup -->
<bean id="sequoia-ds" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<property name="driverClass"
value="org.continuent.sequoia.driver.Driver" />
<property name="jdbcUrl" value="jdbc:sequoia://192.168.100.9/vieo_ccs"
/>
<property name="user" value="postgres" />
<property name="password" value="postgres" />
<property name="minPoolSize" value ="6" />
<property name="maxPoolSize" value="18" />
<property name="acquireIncrement" value="3" />
</bean>
</beans>
--
View this message in context: http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5165212
Sent from the ActiveMQ - User forum at Nabble.com.
Re: AMQ 4.0.1 Memory Leak/Hang
Posted by Sanjiv Jivan <sa...@gmail.com>.
I think backward compatibility for middleware is pretty important (esp. for
different minor releases).
Consider an enterprise where there are several different applications
connecting to a central broker, requiring all clients to upgrade their jars
on a server upgrade might be a concern for some organizations. Oracle 8
and 9 drivers work with Oracle 8, 9 and 10 servers. Similarly I believe RMI
from JDK 1.3 ( and JDK 1.4) is able to talk to RMI from JDK 1.5 although the
internals have changed significantly.
The client server version checking for incompatibilities should be something
that an application may chose to enforce if ,say, the semantics of the
business logic or domain have changed which leaves older clients
incompatible with newer application releases.
On a middleware project that I worked on we had test suites run with older
clients/newer servers as well.
Sanjiv
On 7/5/06, James Strachan <ja...@gmail.com> wrote:
>
> Great - glad to hear its all working fine now :)
>
> Its sometimes hard ensuring the semantics are preserved from release
> to release; I'm wondering if the default behaviour of ActiveMQ should
> be to refuse to connect to a client using another jar version (unless
> explicitly configured to be lenient).
>
> We've a version number in OpenWire we can increase when we make things
> incompatible on the wire format; am wondering if we should at least
> log some kind of warning if (say) a 4.0 client talks to a 4.0.1
> broker as there could be some kind of issue arise. It seems a pretty
> common problem people having a wrong jar on some node in the network
> causing strangeness
>
> On 7/5/06, Dave cawthorn <da...@empired.com> wrote:
> > BTW this use case is a very inefficient use of JMS...
> >
> > I know, it was just a test case to try and expose the problem that i was
> > having. Thanks for the link though, it came in handy for "Discussions"
> with
> > one of the other developers here ;-)
> >
> > Which version of ActiveMQ are you using and can we see your Java code?
> >
> > With some embarrassment I have to admit that my problem appears to have
> been
> > from the fact that my broker was version 4.0.1 but my testing
> environment
> > (eclipse/junit) was still using the 4.0 jar. This was what probably was
> > causing the client to hang. After rectifying that situation I have
> managed
> > to run my test for 2 hours and it appears to be stable now. I'm going to
> run
> > a more comprehensive test using our actual code overnight to see if I've
> > solved the problem i was having.
> >
> > Thanks for your support James!
> >
> > Dave
> > --
> > View this message in context:
> http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5177708
> > Sent from the ActiveMQ - User forum at Nabble.com.
> >
> >
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
Re: AMQ 4.0.1 Memory Leak/Hang
Posted by Dave cawthorn <da...@empired.com>.
Actually it didn't end up being the solution to the problem in my
application :(
However it did lead me to realise that the problem is related to synchronous
messages and specifically i think it may be the temporary queues used to do
the acknowledging.
I have hurriedly created a junit test case that causes my broker to grow in
size untill i get an error message on the client like this:
javax.jms.JMSException: The transport tcp://localhost/127.0.0.1:61616 of
type: org.apache.activemq.transport.tcp.TcpTransport is not running.
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:57)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1122)
at
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1200)
at
org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:271)
at
com.vieo.ccs.activemq.AMQSynchronousTest$1.run(AMQSynchronousTest.java:82)
Caused by: java.io.IOException: The transport
tcp://localhost/127.0.0.1:61616 of type:
org.apache.activemq.transport.tcp.TcpTransport is not running.
at
org.apache.activemq.transport.TransportSupport.checkStarted(TransportSupport.java:108)
at
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:123)
at
org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:141)
at
org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:78)
at
org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:77)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:68)
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:73)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1112)
I was running the broker with its JVM memory size limited to 256M (even
though i saw it grow to 270??) to save a bit of time. When i ran it with
512M i once saw everything stop for a while then suddenly the brokers jvm
size dropped to abot 10M and i saw it log into the database again and then
everything started working again???
Here is the test case:
import javax.jms.Destination;
import javax.jms.Message;
import javax.jms.ObjectMessage;
import javax.jms.TemporaryQueue;
import javax.jms.TextMessage;
import javax.jms.JMSException;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.management.MBeanServer;
import javax.management.ObjectName;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.command.*;
import org.apache.log4j.Logger;
import com.vieo.ccs.common.messages.TestMessage;
import com.vieo.ccs.common.queue.AMQQueueConsumer;
import com.vieo.ccs.common.queue.CommunicationException;
import junit.framework.TestCase;
public class AMQSynchronousTest extends TestCase {
static Logger logger = Logger.getLogger("AMQStressTest");
public void testAMQSendReceive(){
final String connectionString = "tcp://localhost:61616";
ActiveMQConnectionFactory amqFactory = null;
Destination destination = null;
long counter=0;
try {
// get amq connection factory
amqFactory = new ActiveMQConnectionFactory(connectionString);
while(true){
counter++;
logger.info("iteration: "+counter);
amqFactory = new ActiveMQConnectionFactory(connectionString);
ActiveMQConnection producerConnection ;
Session producerSession ;
MessageProducer producer;
producerConnection =
(ActiveMQConnection)amqFactory.createConnection();
producerConnection.setCopyMessageOnSend(false);
producerSession = producerConnection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
if (destination==null){
destination = producerSession.createQueue("MemoryLeakQueue");
}
producer = producerSession.createProducer(destination);
producerConnection.start();
Thread thread = new Thread() {
@Override
public void run() {
try {
ActiveMQConnectionFactory amqFactory2 = new
ActiveMQConnectionFactory(connectionString);
ActiveMQConnection consumerConnection =
(ActiveMQConnection)amqFactory2.createConnection();
Session consumerSession = consumerConnection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
// Not sure if this is part of the problem - normally a jndi lookup
//ActiveMQQueue queue = new ActiveMQQueue("MemoryLeakQueue");
Destination queue = consumerSession.createQueue("MemoryLeakQueue");
MessageConsumer consumer = consumerSession.createConsumer(queue);
consumerConnection.start();
TextMessage msg = (TextMessage)consumer.receive(0);
assertTrue( msg.getText().equals("Test Message") );
Destination tempQueue = (Destination)msg.getJMSReplyTo();
MessageProducer prod = consumerSession.createProducer(tempQueue);
TextMessage reply = consumerSession.createTextMessage("Reply");
prod.send(reply);
tempQueue = null;
prod.close();
prod=null;
consumerSession.close();
consumerSession=null;
consumerConnection.close();
consumerConnection=null;
amqFactory2= null;
} catch (JMSException e) {
e.printStackTrace();
fail();
}
}
};
thread.start();
TemporaryQueue tempQueue = producerSession.createTemporaryQueue();
MessageConsumer tempConsumer = producerSession.createConsumer(tempQueue);
// Create and send the message
TextMessage message= producerSession.createTextMessage("Test Message");
message.setJMSReplyTo(tempQueue);
producer.send(message);
// Wait for the reply
Message reply = tempConsumer.receive(0);
// Delete the temporary consumer and queue
try {
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
tempConsumer.close();
tempQueue.delete();
producerConnection.stop();
producer.close();
producer = null;
producerSession.close();
producerSession = null;
producerConnection.close();
producerConnection = null;
message = null;
amqFactory = null;
thread = null;
System.gc();
}
} catch (JMSException e) {
logger.error("JMS Exception occurred: " + e.getMessage());
fail();
} catch (Exception e){
e.printStackTrace();
fail();
}
}
}
I was a bit rushed so there may be a stupid error in there that is causing
the broker error.
ps I was running this on a windows machine - not sure if this is relevent
but i did notice a lot of connections in a TIME_WAIT state when i did a
netstat while the test was running.
--
View this message in context: http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5197284
Sent from the ActiveMQ - User forum at Nabble.com.
Re: AMQ 4.0.1 Memory Leak/Hang
Posted by James Strachan <ja...@gmail.com>.
Great - glad to hear its all working fine now :)
Its sometimes hard ensuring the semantics are preserved from release
to release; I'm wondering if the default behaviour of ActiveMQ should
be to refuse to connect to a client using another jar version (unless
explicitly configured to be lenient).
We've a version number in OpenWire we can increase when we make things
incompatible on the wire format; am wondering if we should at least
log some kind of warning if (say) a 4.0 client talks to a 4.0.1
broker as there could be some kind of issue arise. It seems a pretty
common problem people having a wrong jar on some node in the network
causing strangeness
On 7/5/06, Dave cawthorn <da...@empired.com> wrote:
> BTW this use case is a very inefficient use of JMS...
>
> I know, it was just a test case to try and expose the problem that i was
> having. Thanks for the link though, it came in handy for "Discussions" with
> one of the other developers here ;-)
>
> Which version of ActiveMQ are you using and can we see your Java code?
>
> With some embarrassment I have to admit that my problem appears to have been
> from the fact that my broker was version 4.0.1 but my testing environment
> (eclipse/junit) was still using the 4.0 jar. This was what probably was
> causing the client to hang. After rectifying that situation I have managed
> to run my test for 2 hours and it appears to be stable now. I'm going to run
> a more comprehensive test using our actual code overnight to see if I've
> solved the problem i was having.
>
> Thanks for your support James!
>
> Dave
> --
> View this message in context: http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5177708
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>
--
James
-------
http://radio.weblogs.com/0112098/
Re: AMQ 4.0.1 Memory Leak/Hang
Posted by Dave cawthorn <da...@empired.com>.
BTW this use case is a very inefficient use of JMS...
I know, it was just a test case to try and expose the problem that i was
having. Thanks for the link though, it came in handy for "Discussions" with
one of the other developers here ;-)
Which version of ActiveMQ are you using and can we see your Java code?
With some embarrassment I have to admit that my problem appears to have been
from the fact that my broker was version 4.0.1 but my testing environment
(eclipse/junit) was still using the 4.0 jar. This was what probably was
causing the client to hang. After rectifying that situation I have managed
to run my test for 2 hours and it appears to be stable now. I'm going to run
a more comprehensive test using our actual code overnight to see if I've
solved the problem i was having.
Thanks for your support James!
Dave
--
View this message in context: http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5177708
Sent from the ActiveMQ - User forum at Nabble.com.
Re: AMQ 4.0.1 Memory Leak/Hang
Posted by James Strachan <ja...@gmail.com>.
BTW this use case is a very inefficient use of JMS...
http://incubator.apache.org/activemq/how-do-i-use-jms-efficiently.html
Which version of ActiveMQ are you using and can we see your Java code?
Are you using networks or temporary destinations or just reusing the
same destination? Using queues/topics/persistence - what
acknoweldgement/transacted mode etc?
On 7/4/06, Dave cawthorn <da...@empired.com> wrote:
>
> Hi,
>
> I have a simple test case that is failing at the moment. It goes like this
>
> 1 create a connection/session/producer
> 2 create a connection/session/consumer
> 3 create a message and send
> 4 readnext message on the consumer
> 5 close every thing
> 6 loop from 1 again
>
> I've tried to configure amq to use persistence so I wont run into memory
> problems (and can support failover with a replicated message store) but when
> i run the test it just consumes memory (amq jvm) till it hangs the client
> when trying to create a session.
>
> here is my config - any ideas?
>
>
> <!-- START SNIPPET: xbean -->
> <beans xmlns="http://activemq.org/config/1.0">
>
> <broker persistent="true" useJmx="false">
>
>
> <memoryManager>
> <usageManager id="memory-manager" limit="100000000"/>
> </memoryManager>
>
>
>
> <managementContext>
> <managementContext connectorPort="1199"
> jmxDomainName="org.apache.activemq"/>
> </managementContext>
>
>
> <!-- In ActiveMQ 4, you can setup destination policies -->
> <destinationPolicy>
> <policyMap><policyEntries>
> <policyEntry topic=">" memoryLimit="50000000"/>
> <policyEntry queue=">" memoryLimit="50000000"/>
> </policyEntries></policyMap>
> </destinationPolicy>
>
>
> <persistenceAdapter>
> <jdbcPersistenceAdapter dataSource="#sequoia-ds" />
> </persistenceAdapter>
>
>
>
> <transportConnectors>
> <transportConnector name="default" uri="tcp://localhost:61616"
> discoveryUri="multicast://default"/>
> <transportConnector name="stomp" uri="stomp://localhost:61613"/>
> </transportConnectors>
>
> <networkConnectors>
> <!-- by default just auto discover the other brokers
> <networkConnector name="default" uri="multicast://default"/> -->
> <!--
> <networkConnector name="host1 and host2"
> uri="static://(tcp://host1:61616,tcp://host2:61616)" failover="true"/>
> -->
> </networkConnectors>
>
> </broker>
>
> <!-- This xbean configuration file supports all the standard spring xml
> configuration options -->
>
> <!-- Postgres DataSource Sample Setup -->
>
>
>
>
>
>
> <bean id="sequoia-ds" class="com.mchange.v2.c3p0.ComboPooledDataSource"
> destroy-method="close">
> <property name="driverClass"
> value="org.continuent.sequoia.driver.Driver" />
> <property name="jdbcUrl" value="jdbc:sequoia://192.168.100.9/vieo_ccs"
> />
> <property name="user" value="postgres" />
> <property name="password" value="postgres" />
> <property name="minPoolSize" value ="6" />
> <property name="maxPoolSize" value="18" />
> <property name="acquireIncrement" value="3" />
> </bean>
>
>
> </beans>
>
> --
> View this message in context: http://www.nabble.com/AMQ-4.0.1-Memory-Leak-Hang-tf1889180.html#a5165212
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>
--
James
-------
http://radio.weblogs.com/0112098/