You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jan Kesten <j....@enercast.de> on 2016/01/12 14:38:57 UTC

ClosedChannelExcption while nodetool repair

Hi,

I have some problems recently on my cassandra cluster. I am running 12
nodes with 2.2.4 and while repairing with a plain "nodetool repair". In
system.log I can find

ERROR [STREAM-IN-/172.17.2.233] 2016-01-08 08:32:38,327
StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
Streaming error occurred
java.nio.channels.ClosedChannelException: null

on one node and at the same time in the the node mentioned in the Log:

INFO  [STREAM-IN-/172.17.2.223] 2016-01-08 08:32:38,073
StreamResultFuture.java:168 - [Stream
#5f96e8b0-b5e2-11e5-b4da-4321ac9959ef ID#0] Prepare completed. Receiving
2 files(46708049 bytes), sending 2 files(1856721742 bytes)
ERROR [STREAM-OUT-/172.17.2.223] 2016-01-08 08:32:38,325
StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
Streaming error occurred
org.apache.cassandra.io.FSReadError: java.io.IOException: Datenübergabe
unterbrochen (broken pipe)
        at
org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144)
~[apache-cassandra-2.2.4.jar:2.2.4]


Full relevant NFO  [STREAM-IN-/172.17.2.223] 2016-01-08 08:32:38,073
StreamResultFuture.java:168 - [Stream
#5f96e8b0-b5e2-11e5-b4da-4321ac9959ef ID#0] Prepare completed. Receiving
2 files(46708049 bytes), sending 2 files(1856721742 bytes)
ERROR [STREAM-OUT-/172.17.2.223] 2016-01-08 08:32:38,325
StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
Streaming error occurred
org.apache.cassandra.io.FSReadError: java.io.IOException: Datenübergabe
unterbrochen (broken pipe)
        at
org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144)
~[apache-cassandra-2.2.4.jar:2.2.4]

More complete log can be found here:

http://pastebin.com/n6DjCCed
http://pastebin.com/6rD5XNwU

I already did a nodetool scrub.

Any suggestions what is causing this?

Thanks in advance,
Jan

Re: ClosedChannelExcption while nodetool repair

Posted by Paulo Motta <pa...@gmail.com>.
You may be running into
https://issues.apache.org/jira/browse/CASSANDRA-10961, which will be fixed
in 2.2.5. In the meantime, you may replace your cassandra jar with a
snapshot version available in that issue.

2016-01-12 10:38 GMT-03:00 Jan Kesten <j....@enercast.de>:

> Hi,
>
> I have some problems recently on my cassandra cluster. I am running 12
> nodes with 2.2.4 and while repairing with a plain "nodetool repair". In
> system.log I can find
>
> ERROR [STREAM-IN-/172.17.2.233] 2016-01-08 08:32:38,327
> StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
> Streaming error occurred
> java.nio.channels.ClosedChannelException: null
>
> on one node and at the same time in the the node mentioned in the Log:
>
> INFO  [STREAM-IN-/172.17.2.223] 2016-01-08 08:32:38,073
> StreamResultFuture.java:168 - [Stream
> #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef ID#0] Prepare completed. Receiving
> 2 files(46708049 bytes), sending 2 files(1856721742 bytes)
> ERROR [STREAM-OUT-/172.17.2.223] 2016-01-08 08:32:38,325
> StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
> Streaming error occurred
> org.apache.cassandra.io.FSReadError: java.io.IOException: Datenübergabe
> unterbrochen (broken pipe)
>         at
> org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144)
> ~[apache-cassandra-2.2.4.jar:2.2.4]
>
>
> Full relevant NFO  [STREAM-IN-/172.17.2.223] 2016-01-08 08:32:38,073
> StreamResultFuture.java:168 - [Stream
> #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef ID#0] Prepare completed. Receiving
> 2 files(46708049 bytes), sending 2 files(1856721742 bytes)
> ERROR [STREAM-OUT-/172.17.2.223] 2016-01-08 08:32:38,325
> StreamSession.java:524 - [Stream #5f96e8b0-b5e2-11e5-b4da-4321ac9959ef]
> Streaming error occurred
> org.apache.cassandra.io.FSReadError: java.io.IOException: Datenübergabe
> unterbrochen (broken pipe)
>         at
> org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144)
> ~[apache-cassandra-2.2.4.jar:2.2.4]
>
> More complete log can be found here:
>
> http://pastebin.com/n6DjCCed
> http://pastebin.com/6rD5XNwU
>
> I already did a nodetool scrub.
>
> Any suggestions what is causing this?
>
> Thanks in advance,
> Jan
>