You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2014/06/11 18:04:02 UTC
[jira] [Created] (SPARK-2111) pyspark errors when
SPARK_PRINT_LAUNCH_COMMAND=1
Thomas Graves created SPARK-2111:
------------------------------------
Summary: pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1
Key: SPARK-2111
URL: https://issues.apache.org/jira/browse/SPARK-2111
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 1.0.0
Reporter: Thomas Graves
If you set SPARK_PRINT_LAUNCH_COMMAND=1 to see what java command is being used to launch spark and then try to run pyspark it errors out with a very non-useful error message:
Traceback (most recent call last):
File "/homes/tgraves/test/hadoop2/y-spark-git/python/pyspark/shell.py", line 43, in <module>
sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
File "/homes/tgraves/test/hadoop2/y-spark-git/python/pyspark/context.py", line 94, in __init__
SparkContext._ensure_initialized(self, gateway=gateway)
File "/homes/tgraves/test/hadoop2/y-spark-git/python/pyspark/context.py", line 184, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway()
File "/homes/tgraves/test/hadoop2/y-spark-git/python/pyspark/java_gateway.py", line 51, in launch_gateway
gateway_port = int(proc.stdout.readline())
ValueError: invalid literal for int() with base 10: 'Spark Command: /home/gs/java/jdk/bin/java -cp :/home/gs/hadoop/current/share/hadoop/common/hadoop-gpl-compression.jar:/home/gs/hadoop/current/share/hadoop/hdfs/lib/YahooDNSToSwitchMapping-0.2.14020207'
--
This message was sent by Atlassian JIRA
(v6.2#6252)