You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2025/05/22 20:28:00 UTC

[jira] [Updated] (SPARK-45891) Support Variant data type

     [ https://issues.apache.org/jira/browse/SPARK-45891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun updated SPARK-45891:
----------------------------------
    Labels: pull-request-available releasenotes  (was: pull-request-available)

> Support Variant data type
> -------------------------
>
>                 Key: SPARK-45891
>                 URL: https://issues.apache.org/jira/browse/SPARK-45891
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 4.0.0
>            Reporter: Chenhao Li
>            Assignee: Chenhao Li
>            Priority: Major
>              Labels: pull-request-available, releasenotes
>             Fix For: 4.0.0
>
>
> I propose to add a Variant data type in Spark. It is used to efficiently represent semi-structured values without a user-specified schema. Currently, many users are depending on JSON expressions to handle JSON data, which can often lead to repeated JSON parsing and degraded performance. One of the major goals of the Variant type is to use a more efficient binary representation internally and avoid repeated JSON parsing. At the same time, it keeps the flexibility of schemaless JSON data.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org