You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Sab <sa...@ibahealth.com> on 2006/07/26 10:10:33 UTC

Connection reset by peer: socket write error

Hi All,

I'm using ActiveMQ 4.0.1 Release and running a thread which sends a simple
Text Message... the idea is just to test the no. of concurrent requests that
the broker can handle.... 

public class LoadTest extends Thread {		
	public static final int THREAD_COUNT = 5000;
	private JmsTemplate jmsTemplate;
	public void run(){						
		try {
			//JMessageCreator is a utility class for creating JMS Messages.
			jmsTemplate.send(new JMessageCreator("Hello World", properties));
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}
	
	public static void main(String args[]) throws IOException{		
		for(int i=0;i<THREAD_COUNT;i++){
			new LoadTest().start();
		}
	}
}

if my thread count is 2500 everything works fine.. but if it exceeds 5000
then the broker is throwing the below exception and subsequently the
consumer crashes and throws org.apache.activemq.ConnectionFailedException:
The JMS connection has failed: java.io.EOFException....

Exception from the broker service:

DEBUG Transport - Transport failed: java.net.SocketException: Connection
reset by peer: socket write error
java.net.SocketException: Connection reset by peer: socket write error
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at
org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:108)
	at java.io.DataOutputStream.flush(DataOutputStream.java:101)
	at
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:125)
	at
org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:141)
	at
org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:78)
	at
org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:77)
	at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
	at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
	at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
	at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:64)
	at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:97)
	at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:63)
	at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:114)
	at
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:122)
	at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:87)
	at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:143)
	at java.lang.Thread.run(Thread.java:536)

I'm trying these thru a Spring's JMSTemplate for sending and
DefaultMessageListenerContainer for processing the messages.... 

Can anyone tell me why its throwing that exception when the no. of requests
increases....  

Thanks in advance!!

Regards
Sab
-- 
View this message in context: http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5499316
Sent from the ActiveMQ - User forum at Nabble.com.


Re: Connection reset by peer: socket write error

Posted by Sab <sa...@ibahealth.com>.
Yes am using PooledConnectionFactory with ActiveMQConnectionFactory....

<bean id="jmsFactory"
class="org.apache.activemq.pool.PooledConnectionFactory">
		<property name="connectionFactory">
			<bean class="org.apache.activemq.ActiveMQConnectionFactory">
				<property name="brokerURL">
					<value>tcp://localhost:61616</value>
				</property>
				<property name="closeTimeout" value="30000"></property>
			</bean>
		</property>
	</bean>
-- 
View this message in context: http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5515322
Sent from the ActiveMQ - User forum at Nabble.com.


Re: Connection reset by peer: socket write error

Posted by James Strachan <ja...@gmail.com>.
Are you using the the PooledConnectionFactory?

http://incubator.apache.org/activemq/jmstemplate-gotchas.html

On 7/26/06, Sab <sa...@ibahealth.com> wrote:
>
> Hi All,
>
> I'm using ActiveMQ 4.0.1 Release and running a thread which sends a simple
> Text Message... the idea is just to test the no. of concurrent requests that
> the broker can handle....
>
> public class LoadTest extends Thread {
>         public static final int THREAD_COUNT = 5000;
>         private JmsTemplate jmsTemplate;
>         public void run(){
>                 try {
>                         //JMessageCreator is a utility class for creating JMS Messages.
>                         jmsTemplate.send(new JMessageCreator("Hello World", properties));
>                 } catch (Exception e) {
>                         // TODO Auto-generated catch block
>                         e.printStackTrace();
>                 }
>         }
>
>         public static void main(String args[]) throws IOException{
>                 for(int i=0;i<THREAD_COUNT;i++){
>                         new LoadTest().start();
>                 }
>         }
> }
>
> if my thread count is 2500 everything works fine.. but if it exceeds 5000
> then the broker is throwing the below exception and subsequently the
> consumer crashes and throws org.apache.activemq.ConnectionFailedException:
> The JMS connection has failed: java.io.EOFException....
>
> Exception from the broker service:
>
> DEBUG Transport - Transport failed: java.net.SocketException: Connection
> reset by peer: socket write error
> java.net.SocketException: Connection reset by peer: socket write error
>         at java.net.SocketOutputStream.socketWrite0(Native Method)
>         at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>         at
> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:108)
>         at java.io.DataOutputStream.flush(DataOutputStream.java:101)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:125)
>         at
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:141)
>         at
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:78)
>         at
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:77)
>         at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
>         at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
>         at
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
>         at
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:64)
>         at
> org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:97)
>         at
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:63)
>         at
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:114)
>         at
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:122)
>         at
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:87)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:143)
>         at java.lang.Thread.run(Thread.java:536)
>
> I'm trying these thru a Spring's JMSTemplate for sending and
> DefaultMessageListenerContainer for processing the messages....
>
> Can anyone tell me why its throwing that exception when the no. of requests
> increases....
>
> Thanks in advance!!
>
> Regards
> Sab
> --
> View this message in context: http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5499316
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>


-- 

James
-------
http://radio.weblogs.com/0112098/

Re: Dropped IP packets

Posted by Aleksi Kallio <al...@csc.fi>.
Hi,

I've found out the reason for this: my messages were too large. So I'll 
have to employ streaming!


> 
> I found this when looking for help to my problem:
> 
> Hiram Chirino wrote:
>> if the OS can't keep up with the ip packets due to full 100% CPU usage it
>> will start to close sockets.  And thus activemq will see jms connections
>> being dropped
> 
> Would dropped IP packets be the reason why I'm getting 
> "javax.jms.JMSException: Connection reset" every time I try to send 
> something substantial?
> 
> This problem arose after optimising my code, which allowed it to process 
> incoming much faster. We are talking about a single large message here.
> 
> Any good ideas how to resolve this?
> 
> 
> My connection string is "tcp://hostname:61616". The full stack trace is:
> 
> javax.jms.JMSException: Connection reset
>         at 
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:45) 
> 
>         at 
> org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1443) 
> 
>         at 
> org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1459) 
> 
>         at 
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94) 
> 
>         at 
> org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:120) 
> 
>         at 
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94) 
> 
>         at 
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94) 
> 
>         at 
> org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120) 
> 
>         at 
> org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149) 
> 
>         at 
> org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:100) 
> 
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:156)
>         at java.lang.Thread.run(Thread.java:595)
> Caused by: java.net.SocketException: Connection reset
>         at java.net.SocketInputStream.read(SocketInputStream.java:168)
>         at 
> org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) 
> 
>         at 
> org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) 
> 
>         at java.io.DataInputStream.readInt(DataInputStream.java:353)
>         at 
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) 
> 
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
>         ... 1 more
> 

Dropped IP packets (was: Re: Connection reset by peer: socket write error)

Posted by Aleksi Kallio <al...@csc.fi>.
I found this when looking for help to my problem:

Hiram Chirino wrote:
> if the OS can't keep up with the ip packets due to full 100% CPU usage it
> will start to close sockets.  And thus activemq will see jms connections
> being dropped

Would dropped IP packets be the reason why I'm getting 
"javax.jms.JMSException: Connection reset" every time I try to send 
something substantial?

This problem arose after optimising my code, which allowed it to process 
incoming much faster. We are talking about a single large message here.

Any good ideas how to resolve this?


My connection string is "tcp://hostname:61616". The full stack trace is:

javax.jms.JMSException: Connection reset
         at 
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:45)
         at 
org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1443)
         at 
org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1459)
         at 
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94)
         at 
org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:120)
         at 
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94)
         at 
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94)
         at 
org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120)
         at 
org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149)
         at 
org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:100)
         at 
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:156)
         at java.lang.Thread.run(Thread.java:595)
Caused by: java.net.SocketException: Connection reset
         at java.net.SocketInputStream.read(SocketInputStream.java:168)
         at 
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
         at 
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
         at java.io.DataInputStream.readInt(DataInputStream.java:353)
         at 
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
         at 
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
         ... 1 more

Re: Connection reset by peer: socket write error

Posted by Hiram Chirino <hi...@hiramchirino.com>.
Hi,

if the OS can't keep up with the ip packets due to full 100% CPU usage it
will start to close sockets.  And thus activemq will see jms connections
being dropped

A better test case would be one that ramps up the load against the server to
figure out what the maximum load your hardware can sustain before failures
start to happen.  To fix this either get more or bigger hardware that will
not be 100% CPU pegged.


On 7/27/06, Sab <sa...@ibahealth.com> wrote:
>
>
> yeah its almost using full CPU time....
>
>
> Hiram Chirino wrote:
> >
> > how long is it at 100% before you start noticing errors?
> >
>
> can't say exactly its varies run to run.... eg. first run - when i started
> from the beginnig the cpu usage is 97-100% and it ran for 5 mins and then
> i
> got the error.....and in the next run few seconds only got the error... in
> that also cpu usage is full.....
>
> but kindly tell me how it is related to the cpu usage?
>
>
> --
> View this message in context:
> http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5515692
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Re: Connection reset by peer: socket write error

Posted by Sab <sa...@ibahealth.com>.
yeah its almost using full CPU time.... 


Hiram Chirino wrote:
> 
> how long is it at 100% before you start noticing errors?
> 

can't say exactly its varies run to run.... eg. first run - when i started
from the beginnig the cpu usage is 97-100% and it ran for 5 mins and then i
got the error.....and in the next run few seconds only got the error... in
that also cpu usage is full..... 

but kindly tell me how it is related to the cpu usage?


-- 
View this message in context: http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5515692
Sent from the ActiveMQ - User forum at Nabble.com.


Re: Connection reset by peer: socket write error

Posted by Hiram Chirino <hi...@hiramchirino.com>.
how does your cpu usage look?  Does it go up to 100% usage and if so, how
long is it at 100% before you start noticing errors?

On 7/26/06, Sab <sa...@ibahealth.com> wrote:
>
>
> Hi All,
>
> I'm using ActiveMQ 4.0.1 Release and running a thread which sends a simple
> Text Message... the idea is just to test the no. of concurrent requests
> that
> the broker can handle....
>
> public class LoadTest extends Thread {
>         public static final int THREAD_COUNT = 5000;
>         private JmsTemplate jmsTemplate;
>         public void run(){
>                 try {
>                         //JMessageCreator is a utility class for creating
> JMS Messages.
>                         jmsTemplate.send(new JMessageCreator("Hello
> World", properties));
>                 } catch (Exception e) {
>                         // TODO Auto-generated catch block
>                         e.printStackTrace();
>                 }
>         }
>
>         public static void main(String args[]) throws IOException{
>                 for(int i=0;i<THREAD_COUNT;i++){
>                         new LoadTest().start();
>                 }
>         }
> }
>
> if my thread count is 2500 everything works fine.. but if it exceeds 5000
> then the broker is throwing the below exception and subsequently the
> consumer crashes and throws org.apache.activemq.ConnectionFailedException:
> The JMS connection has failed: java.io.EOFException....
>
> Exception from the broker service:
>
> DEBUG Transport - Transport failed: java.net.SocketException: Connection
> reset by peer: socket write error
> java.net.SocketException: Connection reset by peer: socket write error
>         at java.net.SocketOutputStream.socketWrite0(Native Method)
>         at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java
> :92)
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>         at
> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(
> TcpBufferedOutputStream.java:108)
>         at java.io.DataOutputStream.flush(DataOutputStream.java:101)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java
> :125)
>         at
> org.apache.activemq.transport.InactivityMonitor.oneway(
> InactivityMonitor.java:141)
>         at
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java
> :78)
>         at
> org.apache.activemq.transport.WireFormatNegotiator.oneway(
> WireFormatNegotiator.java:77)
>         at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java
> :44)
>         at
> org.apache.activemq.transport.ResponseCorrelator.oneway(
> ResponseCorrelator.java:60)
>         at
> org.apache.activemq.broker.TransportConnection.dispatch(
> TransportConnection.java:211)
>         at
> org.apache.activemq.broker.TransportConnection$1.onCommand(
> TransportConnection.java:64)
>         at
> org.apache.activemq.transport.ResponseCorrelator.onCommand(
> ResponseCorrelator.java:97)
>         at
> org.apache.activemq.transport.TransportFilter.onCommand(
> TransportFilter.java:63)
>         at
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(
> WireFormatNegotiator.java:114)
>         at
> org.apache.activemq.transport.InactivityMonitor.onCommand(
> InactivityMonitor.java:122)
>         at
> org.apache.activemq.transport.TransportSupport.doConsume(
> TransportSupport.java:87)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:143)
>         at java.lang.Thread.run(Thread.java:536)
>
> I'm trying these thru a Spring's JMSTemplate for sending and
> DefaultMessageListenerContainer for processing the messages....
>
> Can anyone tell me why its throwing that exception when the no. of
> requests
> increases....
>
> Thanks in advance!!
>
> Regards
> Sab
> --
> View this message in context:
> http://www.nabble.com/Connection-reset-by-peer%3A-socket-write-error-tf2002588.html#a5499316
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com