You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Asif (Jira)" <ji...@apache.org> on 2024/09/18 02:22:00 UTC
[jira] [Updated] (SPARK-49690) UDT type is not expanded into its StructType in the schema definition
[ https://issues.apache.org/jira/browse/SPARK-49690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Asif updated SPARK-49690:
-------------------------
Description:
A UDT type field does not show up as constituent struct type in the schema. Instead it shows up as UserDefinedType class .
For eg: it shows up as
root
|-- message: test (nullable = true)
But mesage is a field of type UDT with schema represented as
StructField("intField", IntegerType, nullable = false),
StructField("stringField", StringType, nullable = false)))
so message field should be a struct type, with schema as
root
|-- message: struct (nullable = true)
| |-- intField: integer (nullable = false)
| |-- stringField: string (nullable = false)
was:A UDT type field does not show up as constituent struct type in the schema. Instead it shows up as UserDefinedType class .
> UDT type is not expanded into its StructType in the schema definition
> ---------------------------------------------------------------------
>
> Key: SPARK-49690
> URL: https://issues.apache.org/jira/browse/SPARK-49690
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Asif
> Priority: Major
> Labels: SQL
>
> A UDT type field does not show up as constituent struct type in the schema. Instead it shows up as UserDefinedType class .
> For eg: it shows up as
> root
> |-- message: test (nullable = true)
>
> But mesage is a field of type UDT with schema represented as
> StructField("intField", IntegerType, nullable = false),
> StructField("stringField", StringType, nullable = false)))
> so message field should be a struct type, with schema as
> root
> |-- message: struct (nullable = true)
> | |-- intField: integer (nullable = false)
> | |-- stringField: string (nullable = false)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org