[jira] [Created] (NIFI-7369) Fixing precision of floating point numbers when reading Avro record

["Simon Bence (Jira)" <ji...@apache.org>]

Simon Bence created NIFI-7369:
---------------------------------

             Summary: Fixing precision of floating point numbers when reading Avro record
                 Key: NIFI-7369
                 URL: https://issues.apache.org/jira/browse/NIFI-7369
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
    Affects Versions: 1.9.2
            Reporter: Simon Bence
            Assignee: Simon Bence
             Fix For: 1.12.0


h3. Summary:

When ConvertRecord reads in decimal (logical) data type from Avro record, it converts it into Java Double, which depending on the decimal precision, might cause precision loss.
h3. In details:

AbstractRecordProcessor (parent of ConvertRecord) works in a way it reads in record with RecordReader and writes it out with RecordSetWriter. Between the two steps, we have an internal representation uses RecordFieldTypes. In this example, the following transformations are happening: AVRO into internal Record. Internal Record into the format defined by the writer. The specific reader where the issue comes up is the AvroReaderWithEmbeddedSchema. This (based on the embedded schema) tries to figure out what could be the proper internal representation for the given fields, using AvroTypeUtil#determineDataType. This util, with the type definition specifies decimal [ends up|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L343-L346] using double (the originally read data with the reader is BigDecimal and it is converted). Double is not capable to correctly represent the incoming data when it is high precision so in further steps, a less accurate number will be used and written into the output.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)