You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2023/11/23 02:05:00 UTC

[jira] [Resolved] (SPARK-45600) Make Python data source registration session level

     [ https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-45600.
----------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Issue resolved by pull request 43742
[https://github.com/apache/spark/pull/43742]

> Make Python data source registration session level
> --------------------------------------------------
>
>                 Key: SPARK-45600
>                 URL: https://issues.apache.org/jira/browse/SPARK-45600
>             Project: Spark
>          Issue Type: Sub-task
>          Components: PySpark
>    Affects Versions: 4.0.0
>            Reporter: Allison Wang
>            Assignee: Allison Wang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>
> Currently, registered data sources are stored in `sharedState` and can be accessed across multiple sessions. This, however, will not work with Spark Connect. We should make this registration session level, and support static registration (e.g. using pip install) in the future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org