[jira] [Commented] (NIFI-9192) Value of QueryDatabaseTableRecord's "Use Avro Logical Types" property ignored when determining the schema of the recordset

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-9192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421013#comment-17421013 ] 

ASF subversion and git services commented on NIFI-9192:
-------------------------------------------------------

Commit 38d26dafa36888826b1017f9f36725628dc40430 in nifi's branch refs/heads/main from Peter Gyori
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=38d26da ]

NIFI-9192: ResultSetRecordSet considers value of useLogicalTypes flag when determining the object's schema.

NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null

Signed-off-by: Peter Gyori <pe...@gmail.com>

NIFI-9192: Unit tests added to test the use of useLogicalType flag

Signed-off-by: Peter Gyori <pe...@gmail.com>

NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER

NIFI-9192: Fixed checkstyle violations

Signed-off-by: Peter Gyori <pe...@gmail.com>

NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes

Signed-off-by: Peter Gyori <pe...@gmail.com>

NIFI-9192: Added unit tests for Array types with and without using logical types

Signed-off-by: Peter Gyori <pe...@gmail.com>

NIFI-9192: Review comments considered and unit tests refactored

NIFI-9192: Added further unit tests

NIFI-9192: Minor modifications based on review comments

Signed-off-by: Peter Gyori <pe...@gmail.com>
Signed-off-by: Matthew Burgess <ma...@apache.org>

This closes #5388


> Value of QueryDatabaseTableRecord's "Use Avro Logical Types" property ignored when determining the schema of the recordset
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-9192
>                 URL: https://issues.apache.org/jira/browse/NIFI-9192
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Peter Gyori
>            Assignee: Peter Gyori
>            Priority: Major
>          Time Spent: 4h
>  Remaining Estimate: 0h
>
> In RecordSqlWriter, the avro schema is determined using this:
> {code:java}
> final Schema avroSchema = JdbcCommon.createSchema(resultSet, options);{code}
> Here, in 'options' the value of "Use Avro Logical Types" is passed and taken into consideration. This avroSchema is used to create the recordAvroSchema, which is OK:
> {code:java}
> final RecordSchema recordAvroSchema = AvroTypeUtil.createSchema(avroSchema);{code}
> *However*, when creating the fullRecordSet object (which contains the schema as well, the creation:
> {code:java}
> fullRecordSet = new ResultSetRecordSetWithCallback(resultSet, recordAvroSchema, callback, options.getDefaultPrecision(), options.getDefaultScale()){code}
> does not consider the value of the useLogicalTypes flag since it is not passed to the constructor. recordAvroSchema was created considering the value of the flag, and is passed to the constructor but *the settings in this schema are overridden during the ResultSetRecordSet creation*, where a schema is created based on the metadata received from the database.
> As a result, if we have "Use Avro Logical Types" set to false, and we have a column in an Oracle DB table (for the sake of this example) that is of type NUMBER, the resulting avro shcema is:
> {code:java}
>     "type" : {
>       "type" : "bytes",
>       "logicalType" : "decimal",
>       "precision" : 10,
>       "scale" : 0
>     }
> {code}
> which unfortunately contains logical avro type.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)