[jira] [Created] (HUDI-7609) Spark cannot write the hudi table containing array type created by flink

["陈磊 (Jira)" <ji...@apache.org>]

陈磊 created HUDI-7609:
------------------------

             Summary: Spark cannot write the hudi table containing array type created by flink
                 Key: HUDI-7609
                 URL: https://issues.apache.org/jira/browse/HUDI-7609
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: 陈磊


When flink creates a Hudi table containing an array field, the elements of the default array field cannot be nullable. When using Spark SQL to read data from the Hive table to the Hudi table, a field verification exception will occur.
{code:java}
2024-03-27 12:47:51 INFO org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object, tree: 'decentral_level1
2024-03-27 12:47:51 INFO at org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute.dataType(unresolved.scala:138)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.types.StructType$.$anonfun$fromAttributes$1(StructType.scala:549)
2024-03-27 12:47:51 INFO at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
2024-03-27 12:47:51 INFO at scala.collection.immutable.List.foreach(List.scala:392)
2024-03-27 12:47:51 INFO at scala.collection.TraversableLike.map(TraversableLike.scala:238)
2024-03-27 12:47:51 INFO at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
2024-03-27 12:47:51 INFO at scala.collection.immutable.List.map(List.scala:298)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.types.StructType$.fromAttributes(StructType.scala:549)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.catalyst.plans.QueryPlan.schema$lzycompute(QueryPlan.scala:281)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.catalyst.plans.QueryPlan.schema(QueryPlan.scala:281)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand$.alignQueryOutput(InsertIntoHoodieTableCommand.scala:153)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand$.run(InsertIntoHoodieTableCommand.scala:105)
2024-03-27 12:47:51 INFO at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand.run(InsertIntoHoodieTableCommand.scala:60) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)