[jira] [Created] (SPARK-23479) struct() cannot be combined with alias(metadata={})

["Tomasz Bartczak (JIRA)" <ji...@apache.org>]

Tomasz Bartczak created SPARK-23479:
---------------------------------------

             Summary: struct() cannot be combined with alias(metadata={})
                 Key: SPARK-23479
                 URL: https://issues.apache.org/jira/browse/SPARK-23479
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.2.1
            Reporter: Tomasz Bartczak


when creating a struct with 'struct()' function - metadata added with alias() is lost:
{code:java}
df =spark.createDataFrame([{'a':1}])
df.select(struct(col('a').alias('a',metadata={'description':'xxx'}))).schema.fields[0].dataType.fields[0].metadata{code}
gives:
{code:java}
{}{code}
workaround is to create the column with metadata before adding it to the struct, but this is obviously bad behaviour:
{code:java}
df =spark.createDataFrame([{'a':1}])
df.select(col('a').alias('a',metadata={'description':'xxx'})).select(struct('a')).schema.fields[0].dataType.fields[0].metadata{code}
keeps metadata:
{code:java}
{'description': 'xxx'}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org