[jira] [Updated] (NIFI-10956) Schema Inference returns incorrect datatype for records where some arrays are empty

["Matt Burgess (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt Burgess updated NIFI-10956:
--------------------------------
    Status: Patch Available  (was: In Progress)

> Schema Inference returns incorrect datatype for records where some arrays are empty
> -----------------------------------------------------------------------------------
>
>                 Key: NIFI-10956
>                 URL: https://issues.apache.org/jira/browse/NIFI-10956
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> If in a FlowFile there is an array field in the schema and for at least one record the value is an empty array and for at least one other record the value is for example a record, the inference logic returns a choice between array<string> and array<record>, and it is possible for the array<string> to be used for the array elements even if they are records. 
> For text-based writers such as JsonRecordSetWriter, this results in a string representation of the record, something like "MapRecord[{a=1,b=2}]" instead of an actual record object. This is a result of empty arrays defaulting to array<string> even if they are part of a choice where there are non-empty arrays. Instead the inference logic should determine if any of the possible choice datatypes are empty arrays and remove them from the list of possible choices (unless that is the only choice, in which case it should default to array<string> as it does now).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)