[jira] [Commented] (SPARK-41919) Unify the schema or datatype in protos

["Martin Grund (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-41919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655346#comment-17655346 ] 

Martin Grund commented on SPARK-41919:
--------------------------------------

There is a compatible way of doing this by adding another field to the oneof that represents the string schema and marking the other fields as reserved.

> Unify the schema or datatype in protos
> --------------------------------------
>
>                 Key: SPARK-41919
>                 URL: https://issues.apache.org/jira/browse/SPARK-41919
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Connect
>    Affects Versions: 3.4.0
>            Reporter: Ruifeng Zheng
>            Priority: Major
>
> this ticket only focus on the protos sent from client to server.
> we normally use 
> {code:java}
>   oneof schema {
>     DataType datatype = 2;
>     // Server will use Catalyst parser to parse this string to DataType.
>     string datatype_str = 3;
>   }
> {code}
> to represent a schema or datatype.
> actually, we can simplify it with just a string. In the server, we can easily parse a DDL-formatted schema or a JSON formatted one.
> {code:java}
>   // (Optional) The schema of local data.
>   // It should be either a DDL-formatted type string or a JSON string.
>   //
>   // The server side will update the column names and data types according to this schema.
>   // If the 'data' is not provided, then this schema will be required.
>   optional string schema = 2;
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org