[jira] [Updated] (SPARK-51975) Support variant_from_arrays and variant_from_entries

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-51975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-51975:
-----------------------------------
    Labels: pull-request-available  (was: )

> Support variant_from_arrays and variant_from_entries 
> -----------------------------------------------------
>
>                 Key: SPARK-51975
>                 URL: https://issues.apache.org/jira/browse/SPARK-51975
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 4.1.0
>            Reporter: Daniël Tom
>            Priority: Major
>              Labels: pull-request-available
>
>  
> Currently, Spark requires a two-step process to create Variant maps from arrays:
>  # Create MapType using map_from_arrays/map_from_entries
>  # Convert to Variant using to_variant_object()
> A direct variant_from_arrays/variant_from_entries implementation reduces memory usage, and improves CPU efficiency. Particularly benefit workloads with large map structures or high-volume of data.
> {*}Example{*}:
> ||Function||Description||Example||
> |variant_from_entries|Transforms an array of key-value pair entries (structs with two fields) into a variant object|{code:java}
> SELECT variant_from_entries(array(struct('1', 'a'), struct('2', 'b')));{code}{code:java}
> {"1":"a", "2":"b"}{code}|
> |variant_from_arrays|Transforms two arrays of
> keys and values respectively, into a new variant object.|{code:java}
> SELECT variant_from_arrays(array('1', '2'), array('a', 'b'));  {code}{code:java}
> {"1":"a", "2":"b"}{code}|
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org