You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ivan Sadikov (Jira)" <ji...@apache.org> on 2022/09/22 07:43:00 UTC
[jira] [Updated] (SPARK-40527) Keep struct field names or map keys in CreateStruct
[ https://issues.apache.org/jira/browse/SPARK-40527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Sadikov updated SPARK-40527:
---------------------------------
Summary: Keep struct field names or map keys in CreateStruct (was: Keep struct field names or map keys in CreateNamedStruct)
> Keep struct field names or map keys in CreateStruct
> ---------------------------------------------------
>
> Key: SPARK-40527
> URL: https://issues.apache.org/jira/browse/SPARK-40527
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Ivan Sadikov
> Priority: Major
>
> Using index-like notation when extracting columns in a struct produces generated column names like {{{}col1{}}}, {{{}col2{}}}, etc. However, when dot-like syntax is used, the column name is correctly selected.
>
> Example:
> {code:java}
> select struct(a['x'], a['y']), struct(a.x, a.y) from (select named_struct('x', 1, 'y', 2) as a) {code}
> Produces the following schema and result:
> {code:java}
> root
> |-- struct(a.x, a.y): struct (nullable = false)
> | |-- col1: integer (nullable = false)
> | |-- col2: integer (nullable = false)
> |-- struct(a.x, a.y): struct (nullable = false)
> | |-- x: integer (nullable = false)
> | |-- y: integer (nullable = false)
> +----------------+----------------+
> |struct(a.x, a.y)|struct(a.x, a.y)|
> +----------------+----------------+
> |{1, 2} |{1, 2} |
> +----------------+----------------+ {code}
>
> It would be good to have {{x}} and {{y}} column names in the first struct.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org