[jira] [Created] (SPARK-56284) Key worker abstraction for language-agnostic UDF protocol

["Haiyang Sun (Jira)" <ji...@apache.org>]

Haiyang Sun created SPARK-56284:
-----------------------------------

             Summary: Key worker abstraction for language-agnostic UDF protocol
                 Key: SPARK-56284
                 URL: https://issues.apache.org/jira/browse/SPARK-56284
             Project: Spark
          Issue Type: Sub-task
          Components: Connect, Spark Core
    Affects Versions: 4.2.0
            Reporter: Haiyang Sun


To enable language-agnostic UDF execution in Spark, a key step is to decouple the Spark engine from language-specific worker runtimes and define a unified interaction model between Spark and those workers.

 

As a first step, this ticket focuses on establishing the core abstractions for describing worker specifications, as well as the primary interfaces Spark will use to interact with workers.

 

This initial step will primarily introduce a new build target/module to host these abstractions. Concrete implementations of the abstractions (e.g., for specific languages or execution backends) will be introduced in follow-up tasks.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org