You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@toree.apache.org by "Mark Gritter (JIRA)" <ji...@apache.org> on 2017/07/24 18:00:00 UTC

[jira] [Created] (TOREE-427) Unable to use display() or magics in Toree-PySpark

Mark Gritter created TOREE-427:
----------------------------------

             Summary: Unable to use display() or magics in Toree-PySpark
                 Key: TOREE-427
                 URL: https://issues.apache.org/jira/browse/TOREE-427
             Project: TOREE
          Issue Type: Bug
          Components: Kernel
    Affects Versions: 0.2.0
         Environment: Linux
            Reporter: Mark Gritter
            Priority: Minor


I am trying to use PySpark in a Jupyter notebook using PySpark.  Plain-text interaction is working fine.  When I tried to include HTML or graphical output, though I get the following:

display(sc.getConf().getAll())
Name: org.apache.toree.interpreter.broker.BrokerException
Message: Traceback (most recent call last):
  File "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py", line 189, in <module>
    eval(compiled_code)
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/display.py", line 300, in display
    format = InteractiveShell.instance().display_formatter.format
  File "/usr/local/lib/python2.7/dist-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 500, in __init__
    self.init_io()
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 660, in init_io
    io.stdout = io.IOStream(sys.stdout)
  File "/usr/local/lib/python2.7/dist-packages/IPython/utils/io.py", line 34, in __init__
    raise ValueError("fallback required, but not specified")
ValueError: fallback required, but not specified

StackTrace: org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
scala.Option.foreach(Option.scala:257)
org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.GatewayConnection.run(GatewayConnection.java:214)
java.lang.Thread.run(Thread.java:748)

I tried using magics to display raw HTML (following the example here: https://github.com/apache/incubator-toree/blob/master/etc/examples/notebooks/magic-tutorial.ipynb) and get the following backtrace:

%%dataframe
Name: org.apache.toree.interpreter.broker.BrokerException
Message: Traceback (most recent call last):
  File "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py", line 189, in <module>
    eval(compiled_code)
  File "<string>", line 1, in <module>
  File "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py", line 107, in __getattr__
    return self._jvm_kernel.__getattribute__(name)
AttributeError: 'JavaObject' object has no attribute 'magics'

StackTrace: org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
scala.Option.foreach(Option.scala:257)
org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.GatewayConnection.run(GatewayConnection.java:214)
java.lang.Thread.run(Thread.java:748)

VERSION: 0.2.0.dev1-incubating
COMMIT: 9b577f19df83




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)