Spark Dataframe write is hanging

[eugene miretsky <eu...@gmail.com>]

Hi,


When I am saving Spark Dataframe to Ignite, the job sometimes get
stuck with the attached error.

It seems to happen at random, and usualy at the end of the job (all
the data has already been writen to Ignite)


Has anybody encountered this before?


java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processConnect(GridNioServer.java:2714)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2333)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2110)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1764)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)

RE: Spark Dataframe write is hanging

[Stanislav Lukyanov <st...@gmail.com>]

Hi,

Were you able to find the root cause of this?
If yes, what was it?

The error indicates a network connection issue, so I guess
the solution should be about the network configuration.

Stan

From: eugene miretsky
Sent: 21 августа 2018 г. 23:35
To: user@ignite.apache.org
Subject: Spark Dataframe write is hanging

Hi, 

When I am saving Spark Dataframe to Ignite, the job sometimes get stuck with the attached error. 
It seems to happen at random, and usualy at the end of the job (all the data has already been writen to Ignite)

Has anybody encountered this before? 

java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processConnect(GridNioServer.java:2714)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2333)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2110)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1764)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)