[jira] [Created] (SPARK-20818) Tab-to-autocomplete in IPython + Python3 results in job execution

["Peter Parente (JIRA)" <ji...@apache.org>]

Peter Parente created SPARK-20818:
-------------------------------------

             Summary: Tab-to-autocomplete in IPython + Python3 results in job execution
                 Key: SPARK-20818
                 URL: https://issues.apache.org/jira/browse/SPARK-20818
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 2.0.2
            Reporter: Peter Parente


Using Spark in a Jupyter Notebook 5.0 with an IPython 6.0 kernel and Python 3, when I press Tab to autocomplete the function names on an RDD, Spark executes the job graph constructed thus far. This only appears to happen for certain autocompletions, namely completions that resolve to functions having the @ignore_unicode_prefix decorator applied to them, and only in Python 3 (never in Python 2). 

https://github.com/apache/spark/blob/master/python/pyspark/rdd.py#L146

Indeed, this function does have a special case for Python 3 in which it rewrites function docstrings. Why IPython autocompletion has a bad interaction with this logic is unknown (to me at least) at the moment.

I reproduced this bug on Spark 2.0.2. The code in the decorator hasn't changed in 2.1.x, so the bug likely impacts that version as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org