[jira] [Assigned] (HUDI-863) nested structs containing decimal types lead to null pointer exception

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan reassigned HUDI-863:
----------------------------------------

    Assignee: Roland Johann

> nested structs containing decimal types lead to null pointer exception
> ----------------------------------------------------------------------
>
>                 Key: HUDI-863
>                 URL: https://issues.apache.org/jira/browse/HUDI-863
>             Project: Apache Hudi (incubating)
>          Issue Type: Bug
>            Reporter: Roland Johann
>            Assignee: Roland Johann
>            Priority: Major
>              Labels: bug-bash-0.6.0, pull-request-available
>             Fix For: 0.6.0
>
>
> Currently the avro schema gets passed to AvroConversionHelper.createConverterToAvro which itself pocesses passed spark sql DataTypes recursively to resolve structs, arrays, etc.  - the AvroSchema gets passed to recursions, but without selection of the relevant field and therefore schema of that field. That leads to a null pointer exception when decimal types will  be processed, because in that case the schema of the filed will be retrieved by calling getField on the root schema which is not defined when we deal with nested records.
> [AvroConversionHelper.scala#L291|https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/scala/org/apache/hudi/AvroConversionHelper.scala#L291]
> The proposed solution is to remove the dependency on the avro schema and derive the particular avro schema for the decimal converter creator case only.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)