You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/03/23 06:06:00 UTC
[jira] [Commented] (SPARK-31152) Issue with Spark Context
Initialization i.e. SparkSession.builder.getOrCreate()
[ https://issues.apache.org/jira/browse/SPARK-31152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064556#comment-17064556 ]
Hyukjin Kwon commented on SPARK-31152:
--------------------------------------
That error isn't from Spark - seems like nothing suggests it's the issue in Spark.
> Issue with Spark Context Initialization i.e. SparkSession.builder.getOrCreate()
> --------------------------------------------------------------------------------
>
> Key: SPARK-31152
> URL: https://issues.apache.org/jira/browse/SPARK-31152
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.4.5, 3.0.0
> Environment: OS : Windows 10
> Spark : spark-3.0.0-preview2-bin-hadoop2.7
>
> Env_variables :
> SPARK_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
> HADOOP_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
> JAVA_HOME : C:\Program Files\Java\jdk1.8.0_191 ; C:\Program Files\Java\jre1.8.0_241\bin
> PYTHON VERSION : Python 3.7.1
> ANACONDA VERSION : conda 4.8.2
>
> I am not running this pyspark code locally (no hadoop setup)- to develop on NLP script.
>
> Reporter: Rohith Bhattaram
> Priority: Major
> Original Estimate: 20h
> Remaining Estimate: 20h
>
> Issue :
> I am trying to initialize spark context using below code :
>
> from pyspark.sql import SparkSession
> spark = SparkSession.builder.getOrCreate()
> import pandas as pd
> sc = spark.sparkContext
>
> At getOrCreate() step , code is going to infinite loop and never respond back either with exception or timeout.
>
> On execution if i check on VS Terminal or Anaconda cmd prompot - below statement is getting displayed :
> [I 19:13:47.973 NotebookApp] Saving file at /LearnPython/Assignment3/NLP.ipynb
> *The filename, directory name, or volume label syntax is incorrect.*
>
>
> Note : This used to work fine till a month back - not sure what chnaged, it stopped working from yesterday.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org