You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Rafal Wojdyla (Jira)" <ji...@apache.org> on 2024/06/25 10:34:00 UTC
[jira] [Created] (SPARK-48711) OOM killer may leave SparkContext in broken state causing ConnectionRefusedError
Rafal Wojdyla created SPARK-48711:
-------------------------------------
Summary: OOM killer may leave SparkContext in broken state causing ConnectionRefusedError
Key: SPARK-48711
URL: https://issues.apache.org/jira/browse/SPARK-48711
Project: Spark
Issue Type: Bug
Components: PySpark, Spark Core
Affects Versions: 3.5.0
Reporter: Rafal Wojdyla
Related to https://issues.apache.org/jira/browse/SPARK-18523, and https://github.com/apache/spark/pull/15961. I'm currently on:
{code}
pyspark 3.5.0 pyhd8ed1ab_0 conda-forge
py4j 0.10.9.7 pyhd8ed1ab_0 conda-forge
{code}
When Spark JVM process gets OOM-Killed, `SparkContext.stop` fails with `ConnectionRefusedError`, which leaves the `SparkSession/Context` in a "dirty" state. https://issues.apache.org/jira/browse/SPARK-18523 addressed this by catching the {{Py4JError}} it looks like the code now raises {{ConnectionRefusedError}}:
{code}
Traceback (most recent call last):
...
File "<TRUNC>/lib/python3.11/site-packages/pyspark/sql/session.py", line 1796, in stop
self._sc.stop()
File "<TRUNC>/lib/python3.11/site-packages/pyspark/context.py", line 654, in stop
self._jsc.stop()
File "<TRUNC>/lib/python3.11/site-packages/py4j/java_gateway.py", line 1321, in __call__
answer = self.gateway_client.send_command(command)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/java_gateway.py", line 1036, in send_command
connection = self._get_connection()
^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 284, in _get_connection
connection = self._create_new_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 291, in _create_new_connection
connection.connect_to_java_server()
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 438, in connect_to_java_server
self.socket.connect((self.java_address, self.java_port))
ConnectionRefusedError: [Errno 111] Connection refused
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org