[jira] [Resolved] (SPARK-35946) Respect Py4J server if InheritableThread API

["Dongjoon Hyun (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-35946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun resolved SPARK-35946.
-----------------------------------
    Fix Version/s: 3.2.0
       Resolution: Fixed

Issue resolved by pull request 33147
[https://github.com/apache/spark/pull/33147]

> Respect Py4J server if InheritableThread API
> --------------------------------------------
>
>                 Key: SPARK-35946
>                 URL: https://issues.apache.org/jira/browse/SPARK-35946
>             Project: Spark
>          Issue Type: Improvement
>          Components: PySpark
>    Affects Versions: 3.2.0
>            Reporter: Hyukjin Kwon
>            Assignee: Hyukjin Kwon
>            Priority: Major
>             Fix For: 3.2.0
>
>
> Currently we sets the enviornment variables at the client side of Py4J (python/pyspark/util.py ). If the Py4J gateway is created somewhere else (e.g., Zeppelin, etc), it could introduce a breakage at:
> {code}
>         from pyspark import SparkContext
>         jvm = SparkContext._jvm
>         thread_connection = jvm._gateway_client.get_thread_connection()
>         # ^ the MLlibMLflowIntegrationSuite test suite failed at this line
>         # `AttributeError: 'GatewayClient' object has no attribute 'get_thread_connection'`
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org