[jira] [Closed] (TOREE-507) Apache Toree - PySpark error

["Luciano Resende (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TOREE-507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luciano Resende closed TOREE-507.
---------------------------------
    Fix Version/s: Not Applicable
         Assignee: Luciano Resende
       Resolution: Won't Fix

We have removed PySpark and SparkR support from Toree.
https://www.mail-archive.com/dev@toree.incubator.apache.org/msg01865.html

Please use IPython, and look at Jupyter Enterprise Gateway for recipes on IPython + Spark.

> Apache Toree - PySpark error 
> -----------------------------
>
>                 Key: TOREE-507
>                 URL: https://issues.apache.org/jira/browse/TOREE-507
>             Project: TOREE
>          Issue Type: Bug
>            Reporter: Aymen ZAGHOUANI
>            Assignee: Luciano Resende
>            Priority: Major
>             Fix For: Not Applicable
>
>
> Hi, I tried all the settings need to run PySpark Toree. Not able to figure out were it went wrong, suggestions!
> Name: org.apache.toree.interpreter.broker.BrokerException
> Message: Traceback (most recent call last):
> File "/tmp/kernel-PySpark-551540d3-c069-4027-a6d8-bc4199a205cd/pyspark_runner.py", line 166, in
> kernel.refreshContext()
> File "/tmp/kernel-PySpark-551540d3-c069-4027-a6d8-bc4199a205cd/pyspark_runner.py", line 131, in refreshContext
> sc = SparkContext(jsc=jsc, gateway=gateway, conf=conf)
> File "/home/likithreddy/spark-2.4.3-bin-hadoop2.7/python/pyspark/context.py", line 127, in *init*
> "You are trying to pass an insecure Py4j gateway to Spark. This"
> ValueError: You are trying to pass an insecure Py4j gateway to Spark. This presents a security risk. If you are sure you understand and accept this risk, you can set the environment variable 'PYSPARK_ALLOW_INSECURE_GATEWAY=1', but note this option will be removed in Spark 3.0
> StackTrace: org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:158)
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:158)
> scala.Option.foreach(Option.scala:257)
> org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:157)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> py4j.Gateway.invoke(Gateway.java:282)
> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> py4j.commands.CallCommand.execute(CallCommand.java:79)
> py4j.GatewayConnection.run(GatewayConnection.java:238)
> java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)