pyspark does not seem to start py4j callback server

[girishlg <gi...@gmail.com>]

Hi

We have a use case where we call a scala function with a python object as a
callback. The python object implements a scala trait. The call to scala
function goes through but when it makes a call back through the passed in
python object we get a connection refused error. Looking further we noticed
the java_gateway.py launch_gateway method initiates the gateway object with
this line 

# Connect to the gateway
   gateway = JavaGateway(GatewayClient(port=gateway_port),
auto_convert=True)

and as per https://www.py4j.org/py4j_java_gateway.html the callback server
is started only if callback_server_parameters is passed.

Could someone please help us understand if in fact the callback server is
not started and if so any particular reason why it is disabled?

Thanks for any pointers
Girish




--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/pyspark-does-not-seem-to-start-py4j-callback-server-tp15341.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org