You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2026/07/20 14:38:00 UTC
[jira] [Resolved] (SPARK-58162) Spark Connect client-side unbounded RetryException retry loop in reattach path can hang forever
[ https://issues.apache.org/jira/browse/SPARK-58162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-58162.
---------------------------------
Fix Version/s: 4.3.0
Resolution: Fixed
Issue resolved by pull request 57320
[https://github.com/apache/spark/pull/57320]
> Spark Connect client-side unbounded RetryException retry loop in reattach path can hang forever
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-58162
> URL: https://issues.apache.org/jira/browse/SPARK-58162
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 4.1.0, 4.2.0, 4.3.0, 5.0.0
> Reporter: Vinod KC
> Assignee: Vinod KC
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.3.0
>
> Attachments: spark-connect-retryexception-hang-demo.zip
>
>
> This is a Spark Connect client-side issue . The bug is entirely in the client's local retry/reattach handling logic (both the Scala and Python Connect clients).
> Spark Connect's client-side reattach logic retries an internal error marker (RetryException) with no limit — no retry count cap, no backoff, and no time bound.
> This marker is raised client-side when a reattach attempt keeps hitting `DEADLINE_EXCEEDED`, or when the initial query
> never reached the server at all.
> If the connection to the server becomes unable to make progress (e.g. a load balancer or NAT gateway silently drops it, or gRPC keepalive is disabled or hasn't detected the dead connection yet), the client retries this cycle forever, entirely on its own — the server is stateless in this exchange and does nothing wrong. The client never returns and never throws — a permanent hang, affecting both the Scala and Python clients identically.
> A minimal, runnable reproduction is attached: `spark-connect-retryexception-hang-demo.zip` a real Spark Connect server, a separate client process, and a small proxy that can be frozen to simulate a dropped connection. A thread dump confirms the client is permanently stuck retrying inside its own reattach retry loop, with no way out, while the server remains idle and unaffected throughout.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org