You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ga...@apache.org on 2026/07/22 20:54:09 UTC
(spark) branch branch-4.x updated: [SPARK-58268][PYTHON] Disable GC for Config gRPC to avoid deadlock
This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 4db24399c070 [SPARK-58268][PYTHON] Disable GC for Config gRPC to avoid deadlock
4db24399c070 is described below
commit 4db24399c070532d46b2ce275ae1a94856902553
Author: Tian Gao <ga...@hotmail.com>
AuthorDate: Wed Jul 22 13:53:45 2026 -0700
[SPARK-58268][PYTHON] Disable GC for Config gRPC to avoid deadlock
### What changes were proposed in this pull request?
Disable GC when we do `_stub.Config()`.
### Why are the changes needed?
This is the same thing for https://github.com/apache/spark/pull/53783 . We need to disable GC when we do gRPC to avoid deadlock. We only recently found this method due to https://github.com/apache/spark/actions/runs/29869961676 .
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
CI.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57416 from gaogaotiantian/fix-config-deadlock.
Authored-by: Tian Gao <ga...@hotmail.com>
Signed-off-by: Tian Gao <ga...@hotmail.com>
(cherry picked from commit 377a00e2a047d85bff71a03d3e929b8f0022faed)
Signed-off-by: Tian Gao <ga...@hotmail.com>
---
python/pyspark/sql/connect/client/core.py | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/python/pyspark/sql/connect/client/core.py b/python/pyspark/sql/connect/client/core.py
index f0f1a23789dc..be42e8259e83 100644
--- a/python/pyspark/sql/connect/client/core.py
+++ b/python/pyspark/sql/connect/client/core.py
@@ -2057,11 +2057,12 @@ class SparkConnectClient(object):
try:
for attempt in self._retrying():
with attempt:
- resp = self._stub.Config(
- req,
- metadata=self._builder.metadata(),
- timeout=self._rpc_deadlines.config,
- )
+ with disable_gc():
+ resp = self._stub.Config(
+ req,
+ metadata=self._builder.metadata(),
+ timeout=self._rpc_deadlines.config,
+ )
self._verify_response_integrity(resp)
return ConfigResult.fromProto(resp)
raise SparkConnectException("Invalid state during retry exception handling.")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org