You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/12/15 09:17:58 UTC
[jira] [Resolved] (SPARK-18876) An error occurred while trying to
connect to the Java server
[ https://issues.apache.org/jira/browse/SPARK-18876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-18876.
-------------------------------
Resolution: Duplicate
This doesn't actually say anything about the problem -- just says "something went wrong". This is likely to be caused by something else, elaborated in JIRAs like SPARK-18523
> An error occurred while trying to connect to the Java server
> ------------------------------------------------------------
>
> Key: SPARK-18876
> URL: https://issues.apache.org/jira/browse/SPARK-18876
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.0.1, 2.0.2
> Environment: Python 2.7.12
> Reporter: pranludi
>
> I am trying to create spark context object with the following commands on pyspark:
> ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35918)
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 963, in start
> self.socket.connect((self.address, self.port))
> File "/usr/local/lib/python2.7/socket.py", line 228, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 111] Connection refused
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/home/gamedev/spark-2.0.1-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 419, in coalesce
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 1131, in __call__
> answer = self.gateway_client.send_command(command)
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 881, in send_command
> connection = self._get_connection()
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 829, in _get_connection
> connection = self._create_connection()
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 835, in _create_connection
> connection.start()
> File "/usr/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 970, in start
> raise Py4JNetworkError(msg, e)
> py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:35918)
> ---------------------------------------------
> I try spark version 2.0.0, 2.0.1, 2.0.2
> no problem 2.0.0
> but 2.0.1, 2.0.2 occur
> python code
> ------
> .....
> df = spark.read.json('hdfs://big_big_4000000.json')
> json_log = []
> for log in df.collect():
> jj = {}
> try:
> for f in log.__fields__:
> if f == 'I_LogDes':
> if log[f] is not None:
> log_des_json = json.loads(log[f])
> for jf in log_des_json:
> json_key = add_2(jf)
> if json_key in jj:
> json_key = '%s_2' % json_key
> jj[json_key] = typeIntStr(log_des_json[jf])
> else:
> jj[remove_i(f)] = typeIntStr(log[f])
> json_log.append(jj)
> except:
> print log
> # !!! here error occur
> df = spark.read.json(spark.sparkContext.parallelize(json_log))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org