[jira] [Created] (NIFI-9256) QueryDatabaseTable Processor Default Decimal Scale not working

["GorkemG (Jira)" <ji...@apache.org>]

GorkemG created NIFI-9256:
-----------------------------

             Summary: QueryDatabaseTable Processor Default Decimal Scale not working
                 Key: NIFI-9256
                 URL: https://issues.apache.org/jira/browse/NIFI-9256
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
    Affects Versions: 1.14.0
         Environment: Linux HOSTNAME 4.18.0-305.19.1.el8_4.x86_64 #1 SMP Tue Sep 7 07:07:31 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux

openjdk version "1.8.0_302"
OpenJDK Runtime Environment (build 1.8.0_302-b08)
OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)


            Reporter: GorkemG


When using QueryDatabaseTable processor , and to infer schemea from table set Use Avro Logical Types = True,  whether you change Default Decimal Precision value to other than default 10 or not, and set Default Decimal Scale to desired number (other than default 0) , in the output avro schema scale is "always" set to whatever Default Decimal Precision is.

E.g. if Default Decimal Precision = 38 -> Default Decimal Scale is always 38  (irrelevant to value)

On the other hand ExecuteSQL Processor is working as intended, changing scale and precision affects avro schema (using same OracleSQL connection).

Note that QueryDatabaseTable is handy for incremental fetching, and if Avro Logical Types is false (default) columns are always string type and when you convert to for example Parquet, you miss the schema types and there is no handy way to pass out that information.

Usually when reading Parquet files Precision=38 and Scale=18 is required (due to other software conversion missing features).

 

Using QueryDatabaseTable

"fields":[{"name":"ID","type":["null",{"type":"bytes","logicalType":"decimal","precision":38,*"scale":38*}]}

 

Using ExecuteSQL

{"name":"ID","type":["null",\{"type":"bytes","logicalType":"decimal","precision":38,"scale":18}]}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)