You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jacob Rhoden <ja...@me.com> on 2014/07/01 00:29:54 UTC
Re: Connection reset by peer error
How are the two machines connected? Direct cable? Via a hub, router, firewall, wan?
______________________________
Sent from iPhone
> On 1 Jul 2014, at 6:01 am, cass savy <ca...@gmail.com> wrote:
>
> We use Datastax Java driver version 1.0.6. Application is running into issues connecting to the 3 node cluster. What is cause for it? Application is not able to establish a connection at all. I see this error intermittently few time every other day.
>
>
> Is the issue related to read/write timeout?Do I need to increase *timeout* values in yaml ?
> APP logs
> 2014-06-27 17:33:47
>
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode):
>
>
>
> "RMI TCP Connection(105)-10.198.49.16" - Thread t@247
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:150)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:121)
>
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>
> - locked <68d37818> (a java.io.BufferedInputStream)
>
> at java.io.FilterInputStream.read(FilterInputStream.java:83)
>
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
>
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>
> at java.lang.Thread.run(Thread.java:722)
>
>
>
>
>
> All I see in the Cassandra logs:
> ERROR [Native-Transport-Requests:2704] 2014-06-27 16:33:23,339 ErrorMessage.java (line 210) Unexpected exception during request
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcher.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(Unknown Source)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.read(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:59)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
>
Re: Connection reset by peer error
Posted by Jacob Rhoden <ja...@me.com>.
The information about how the servers are connected is important, because we have exactly these types of situations in some of our applications (not using Cassandra) when firewall administrators/configurators get “creative” about “enhancing” security. Other things can cause this type of situation, but in my limited experience, I’ve only ever seen it caused by the firewall.
Best regards,
Jacob
On 1 Jul 2014, at 12:55 pm, cass savy <ca...@gmail.com> wrote:
> The app and Cassandra are connected via firewall. For some reason, connections are still remaining on Cassandra side even after stopping services on app server.
>
> On Mon, Jun 30, 2014 at 3:29 PM, Jacob Rhoden <ja...@me.com> wrote:
> How are the two machines connected? Direct cable? Via a hub, router, firewall, wan?
>
> On 1 Jul 2014, at 6:01 am, cass savy <ca...@gmail.com> wrote:
>> We use Datastax Java driver version 1.0.6. Application is running into issues connecting to the 3 node cluster. What is cause for it? Application is not able to establish a connection at all. I see this error intermittently few time every other day.
>>
>>
>> Is the issue related to read/write timeout?Do I need to increase *timeout* values in yaml ?
>> APP logs
>> 2014-06-27 17:33:47
>>
>> Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode):
>>
>>
>> "RMI TCP Connection(105)-10.198.49.16" - Thread t@247
>>
>> java.lang.Thread.State: RUNNABLE
>>
>> at java.net.SocketInputStream.socketRead0(Native Method)
>>
>> at java.net.SocketInputStream.read(SocketInputStream.java:150)
>>
>> at java.net.SocketInputStream.read(SocketInputStream.java:121)
>>
>> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>
>> at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>>
>> - locked <68d37818> (a java.io.BufferedInputStream)
>>
>> at java.io.FilterInputStream.read(FilterInputStream.java:83)
>>
>> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>
>> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
>>
>> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
>>
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>
>> at java.lang.Thread.run(Thread.java:722)
>>
>>
>>
>>
>>
>> All I see in the Cassandra logs:
>> ERROR [Native-Transport-Requests:2704] 2014-06-27 16:33:23,339 ErrorMessage.java (line 210) Unexpected exception during request
>> java.io.IOException: Connection reset by peer
>> at sun.nio.ch.FileDispatcher.read0(Native Method)
>> at sun.nio.ch.SocketDispatcher.read(Unknown Source)
>> at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
>> at sun.nio.ch.IOUtil.read(Unknown Source)
>> at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
>> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:59)
>> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472)
>> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333)
>> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> at java.lang.Thread.run(Unknown Source)
>>
>>
>
Re: Connection reset by peer error
Posted by cass savy <ca...@gmail.com>.
The app and Cassandra are connected via firewall. For some reason,
connections are still remaining on Cassandra side even after stopping
services on app server.
On Mon, Jun 30, 2014 at 3:29 PM, Jacob Rhoden <ja...@me.com> wrote:
> How are the two machines connected? Direct cable? Via a hub, router,
> firewall, wan?
>
> ______________________________
> Sent from iPhone
>
> On 1 Jul 2014, at 6:01 am, cass savy <ca...@gmail.com> wrote:
>
> We use Datastax Java driver version 1.0.6. Application is running into
> issues connecting to the 3 node cluster. What is cause for it? Application
> is not able to establish a connection at all. I see this error
> intermittently few time every other day.
>
>
> Is the issue related to read/write timeout?Do I need to increase *timeout*
> values in yaml ?
> *APP logs*
>
> 2014-06-27 17:33:47
>
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode):
>
>
>
> "RMI TCP Connection(105)-10.198.49.16" - Thread t@247
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at
> java.net.SocketInputStream.read(SocketInputStream.java:150)
>
> at
> java.net.SocketInputStream.read(SocketInputStream.java:121)
>
> at
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>
> at
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>
> - locked <68d37818> (a java.io.BufferedInputStream)
>
> at
> java.io.FilterInputStream.read(FilterInputStream.java:83)
>
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>
> at java.lang.Thread.run(Thread.java:722)
>
>
>
> *All I see in the Cassandra logs:*
> ERROR [Native-Transport-Requests:2704] 2014-06-27 16:33:23,339
> ErrorMessage.java (line 210) Unexpected exception during request
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcher.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(Unknown Source)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.read(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:59)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Unknown Source)
>
>
>