You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2014/07/02 02:21:34 UTC

[jira] [Resolved] (SPARK-2003) SparkContext(SparkConf) doesn't work in pyspark

     [ https://issues.apache.org/jira/browse/SPARK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Patrick Wendell resolved SPARK-2003.
------------------------------------

    Resolution: Won't Fix

> SparkContext(SparkConf) doesn't work in pyspark
> -----------------------------------------------
>
>                 Key: SPARK-2003
>                 URL: https://issues.apache.org/jira/browse/SPARK-2003
>             Project: Spark
>          Issue Type: Bug
>          Components: Documentation, PySpark
>    Affects Versions: 1.0.0
>            Reporter: Diana Carroll
>             Fix For: 1.0.1, 1.1.0
>
>
> Using SparkConf with SparkContext as described in the Programming Guide does NOT work in Python:
> conf = SparkConf.setAppName("blah")
> sc = SparkContext(conf)
> When I tried I got 
> AttributeError: 'SparkConf' object has no attribute '_get_object_id'
> [This equivalent code in Scala works fine:
> val conf = new SparkConf().setAppName("blah")
> val sc = new SparkContext(conf)]
> I think this is because there's no equivalent for the Scala constructor SparkContext(SparkConf).  
> Workaround:
> If I explicitly set the conf parameter in the python call, it does work:
> sconf = SparkConf.setAppName("blah")
> sc = SparkContext(conf=sconf)



--
This message was sent by Atlassian JIRA
(v6.2#6252)