You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Herman van Hövell (Jira)" <ji...@apache.org> on 2024/11/25 13:54:00 UTC
[jira] [Updated] (SPARK-50411) Spark Connect the default API in Spark 4
[ https://issues.apache.org/jira/browse/SPARK-50411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Herman van Hövell updated SPARK-50411:
--------------------------------------
Summary: Spark Connect the default API in Spark 4 (was: We to make Spark Connect the default API in Spark 4)
> Spark Connect the default API in Spark 4
> ----------------------------------------
>
> Key: SPARK-50411
> URL: https://issues.apache.org/jira/browse/SPARK-50411
> Project: Spark
> Issue Type: Epic
> Components: Connect, SQL
> Affects Versions: 4.0.0
> Reporter: Herman van Hövell
> Assignee: Herman van Hövell
> Priority: Major
>
> We propose to enable Spark Connect as the default interface for Scala and Python applications in Spark 4. With this change, we will ensure that by default, all Scala and Python applications will run using the Spark Connect API so that developers can 1) benefit from stability, upgradability, and ease of use and 2) use Spark from non-JVM languages.
> As a lot of cases are using the full Spark API, we want to make it simple to switch back for compatibility reasons. We propose to add a new Spark configuration spark.api.mode with the following options:
> * connect: Spark uses the Spark Connect as the default API (default)
> * classic: Spark uses classic mode. The classic mode is today’s default.
> With this configuration, we open up Spark to a new generation of developers while ensuring all existing workloads continue to run without any impact!
> SPIP: https://docs.google.com/document/d/1C0kuQEliG78HujVwdnSk0wjNwHEDdwo2o8aVq7kbhTo/edit?tab=t.0#heading=h.r2c3xrbiklu3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org