You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Hahn Jiang <ha...@gmail.com> on 2014/04/11 09:55:32 UTC
Error when I use spark-streaming
hi all,
When I run spark-streaming use NetworkWordCount in example, it always
throw this Exception. I don't understand why it can't connect and I don't
restrict 9999 port.
14/04/11 15:38:56 ERROR SocketReceiver: Error receiving data in receiver 0
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at
org.apache.spark.streaming.dstream.SocketReceiver.onStart(SocketInputDStream.scala:57)
at
org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:147)
at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:201)
at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:197)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
at org.apache.spark.scheduler.Task.run(Task.scala:52)
at
org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:211)
at
org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:176)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
14/04/11 15:38:56 ERROR NetworkInputTracker: Deregistered receiver for
network stream 0 with message:
Thanks
Re: Error when I use spark-streaming
Posted by Hahn Jiang <ha...@gmail.com>.
I found it. I should run "nc -lk 9999" at first and then run the
NetworkWordCount.
Thanks
On Fri, Apr 11, 2014 at 4:13 PM, Schein, Sagi <sa...@hp.com> wrote:
> I would check the DNS setting.
>
> Akka seems to pick configuration from FQDN on my system
>
>
>
> Sagi
>
>
>
> *From:* Hahn Jiang [mailto:hahn.jiang.pro@gmail.com]
> *Sent:* Friday, April 11, 2014 10:56 AM
> *To:* user
> *Subject:* Error when I use spark-streaming
>
>
>
> hi all,
>
> When I run spark-streaming use NetworkWordCount in example, it always
> throw this Exception. I don't understand why it can't connect and I don't
> restrict 9999 port.
>
>
>
> 14/04/11 15:38:56 ERROR SocketReceiver: Error receiving data in receiver 0
>
> java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:579)
>
> at java.net.Socket.connect(Socket.java:528)
>
> at java.net.Socket.<init>(Socket.java:425)
>
> at java.net.Socket.<init>(Socket.java:208)
>
> at
> org.apache.spark.streaming.dstream.SocketReceiver.onStart(SocketInputDStream.scala:57)
>
> at
> org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:147)
>
> at
> org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:201)
>
> at
> org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:197)
>
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
>
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
>
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
>
> at org.apache.spark.scheduler.Task.run(Task.scala:52)
>
> at
> org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:211)
>
> at
> org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
>
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:176)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
> 14/04/11 15:38:56 ERROR NetworkInputTracker: Deregistered receiver for
> network stream 0 with message:
>
>
>
> Thanks
>
RE: Error when I use spark-streaming
Posted by "Schein, Sagi" <sa...@hp.com>.
I would check the DNS setting.
Akka seems to pick configuration from FQDN on my system
Sagi
From: Hahn Jiang [mailto:hahn.jiang.pro@gmail.com]
Sent: Friday, April 11, 2014 10:56 AM
To: user
Subject: Error when I use spark-streaming
hi all,
When I run spark-streaming use NetworkWordCount in example, it always throw this Exception. I don't understand why it can't connect and I don't restrict 9999 port.
14/04/11 15:38:56 ERROR SocketReceiver: Error receiving data in receiver 0
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at org.apache.spark.streaming.dstream.SocketReceiver.onStart(SocketInputDStream.scala:57)
at org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:147)
at org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:201)
at org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$9.apply(NetworkInputTracker.scala:197)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1042)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
at org.apache.spark.scheduler.Task.run(Task.scala:52)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:211)
at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:176)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
14/04/11 15:38:56 ERROR NetworkInputTracker: Deregistered receiver for network stream 0 with message:
Thanks