[jira] [Created] (SPARK-53307) [CONNECT][CLIENT][PYTHON][SCALA] Remove RetriesExceeded error from Spark Connect Python and Scala clients

["Alex Khakhlyuk (Jira)" <ji...@apache.org>]

Alex Khakhlyuk created SPARK-53307:
--------------------------------------

             Summary: [CONNECT][CLIENT][PYTHON][SCALA] Remove RetriesExceeded error from Spark Connect Python and Scala clients
                 Key: SPARK-53307
                 URL: https://issues.apache.org/jira/browse/SPARK-53307
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: 4.1.0
            Reporter: Alex Khakhlyuk
             Fix For: 4.1.0


RetriesExceeded was introduced in [#43591|https://github.com/apache/spark/pull/43591] 2 years ago.
This exception turned out to be more harmful than useful.
 # The user does not see the underlying issue, only {{[RETRIES_EXCEEDED] The maximum number of retries has been exceeded.}} To get the underlying root exception, they need to inspect {{{}RetriesExceeded.__cause__{}}}, which is a bad interface.
 # {{RetriesExceeded}} is a {{{}PySparkException{}}}, which means that it and the root cause are NOT handled by the error handling logic for spark connect grpc exceptions in [_handle_rpc_error|https://github.com/apache/spark/blob/337a67ff58ad91f2f86751bcb9a5e50e1de5cef2/python/pyspark/sql/connect/client/core.py#L1833].
 # {{RetriesExceeded}} is not used anywhere in the code base except throwing it in retries.py.
 # To simplify error handling, we replace throwing RetriesExceeded by throwing the original error. We also remove all internal references to RetriesExceeded in the code base.

Description above discusses Python code, the same applies for Scala.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org