You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ralf Steppacher <ra...@gmail.com> on 2016/05/23 13:48:10 UTC

sstableloader: Stream failed

Hello,

I am trying to load the SSTables (from a Titan graph keyspace) of a one-node-cluster (C* v2.2.6) into another node, but I cannot figure out how to properly use the sstableloader. The target keyspace and table exist in the target node. If they do not exist I get a proper error message telling me so. 
Providing a cassandra.yaml or not makes no difference. 
The listen_address and rpc_address values in the cassandra.yaml, if provided, do not seem to matter (at least the error is always the same).
Running sstableloader on the C* node itself or another host makes no difference.
Truncating all tables before attempting to load the date makes no difference.

The node is up and running:
INFO  13:41:18 Starting listening for CQL clients on /10.211.55.8:9042...
INFO  13:41:18 Binding thrift service to /10.211.55.8:9160
INFO  13:41:18 Listening for thrift clients...


The error I am getting is this:

$ ./sstableloader -d 10.211.55.8 -f ../conf/cassandra.yaml -v ~/Downloads/
ams0002-cassandra-20160523-1035/var/lib/cassandra/data/Titan/edgestore-8bcd2300d0d011e5a3ab233f92747e94/
objc[18941]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Established connection to initial hosts
Opening sstables and calculating sections to stream
Streaming relevant part of /Users/rsteppac/Downloads/ams0002-cassandra-20160523-1035/var/lib/cassandra/data/Titan/edgestore-8bcd2300d0d011e5a3ab233f92747e94/la-1-big-Data.db to [/10.211.55.8]
ERROR 12:57:24 [Stream #e4b9cbc0-20e5-11e6-a00f-4b867a050904] Streaming error occurred
java.net.ConnectException: Connection refused
	at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77]
	at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77]
	at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77]
	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) ~[na:1.8.0_77]
	at java.nio.channels.SocketChannel.open(SocketChannel.java:189) ~[na:1.8.0_77]
	at org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:248) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:83) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:235) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212) [apache-cassandra-2.2.6.jar:2.2.6]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
progress: total: 100% 0  MB/s(avg: 0 MB/s)WARN  12:57:24 [Stream #e4b9cbc0-20e5-11e6-a00f-4b867a050904] Stream failed
Streaming to the following hosts failed:
[/10.211.55.8]
java.util.concurrent.ExecutionException: org.apache.cassandra.streaming.StreamException: Stream failed
	at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
	at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:115)
Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
	at org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
	at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172)
	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
	at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
	at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
	at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:186)
	at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:434)
	at org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:529)
	at org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:241)
	at org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


If anyone could point out where I am going wrong I’d appreciate it!

Thanks
Ralf

Re: sstableloader: Stream failed

Posted by Ralf Steppacher <ra...@gmail.com>.
Thanks for the hint! Indeed I could not telnet to the host. It was the listen_address that was not properly configured.

Thanks again!
Ralf


> On 23.05.2016, at 21:01, Paulo Motta <pa...@gmail.com> wrote:
> 
> Can you telnet 10.211.55.8 7000? This is the port used for streaming communication with the destination node.
> 
> If not you should check what is the configured storage_port in the destination node and set that in the cassandra.yaml of the source node so it's picked up by sstableloader.
> 


Re: sstableloader: Stream failed

Posted by Paulo Motta <pa...@gmail.com>.
Can you telnet 10.211.55.8 7000? This is the port used for streaming
communication with the destination node.

If not you should check what is the configured storage_port in the
destination node and set that in the cassandra.yaml of the source node so
it's picked up by sstableloader.

2016-05-23 10:48 GMT-03:00 Ralf Steppacher <ra...@gmail.com>:

> Hello,
>
> I am trying to load the SSTables (from a Titan graph keyspace) of a
> one-node-cluster (C* v2.2.6) into another node, but I cannot figure out how
> to properly use the sstableloader. The target keyspace and table exist in
> the target node. If they do not exist I get a proper error message telling
> me so.
> Providing a cassandra.yaml or not makes no difference.
> The listen_address and rpc_address values in the cassandra.yaml, if
> provided, do not seem to matter (at least the error is always the same).
> Running sstableloader on the C* node itself or another host makes no
> difference.
> Truncating all tables before attempting to load the date makes no
> difference.
>
> The node is up and running:
> INFO  13:41:18 Starting listening for CQL clients on /10.211.55.8:9042...
> INFO  13:41:18 Binding thrift service to /10.211.55.8:9160
> INFO  13:41:18 Listening for thrift clients...
>
>
> The error I am getting is this:
>
> $ ./sstableloader -d 10.211.55.8 -f ../conf/cassandra.yaml -v ~/Downloads/
>
> ams0002-cassandra-20160523-1035/var/lib/cassandra/data/Titan/edgestore-8bcd2300d0d011e5a3ab233f92747e94/
> objc[18941]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/bin/java
> and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> One of the two will be used. Which one is undefined.
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /Users/rsteppac/Downloads/ams0002-cassandra-20160523-1035/var/lib/cassandra/data/Titan/edgestore-8bcd2300d0d011e5a3ab233f92747e94/la-1-big-Data.db
> to [/10.211.55.8]
> ERROR 12:57:24 [Stream #e4b9cbc0-20e5-11e6-a00f-4b867a050904] Streaming
> error occurred
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_77]
>         at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_77]
>         at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_77]
>         at
> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_77]
>         at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_77]
>         at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[apache-cassandra-2.2.6.jar:2.2.6]
>         at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:248)
> ~[apache-cassandra-2.2.6.jar:2.2.6]
>         at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:83)
> ~[apache-cassandra-2.2.6.jar:2.2.6]
>         at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:235)
> ~[apache-cassandra-2.2.6.jar:2.2.6]
>         at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [apache-cassandra-2.2.6.jar:2.2.6]
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_77]
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_77]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> progress: total: 100% 0  MB/s(avg: 0 MB/s)WARN  12:57:24 [Stream
> #e4b9cbc0-20e5-11e6-a00f-4b867a050904] Stream failed
> Streaming to the following hosts failed:
> [/10.211.55.8]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
>         at
> com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
>         at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
>         at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
>         at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:115)
> Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
>         at
> org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
>         at
> com.google.common.util.concurrent.Futures$4.run(Futures.java:1172)
>         at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>         at
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
>         at
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
>         at
> com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
>         at
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
>         at
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:186)
>         at
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:434)
>         at
> org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:529)
>         at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:241)
>         at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> If anyone could point out where I am going wrong I’d appreciate it!
>
> Thanks
> Ralf