[jira] [Reopened] (NIFI-4473) Add support for large result sets and normalizing Avro names to SelectHiveQL

["Matt Burgess (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt Burgess reopened NIFI-4473:
--------------------------------

Reopening due to issue(s) with error handling. It is possible for the content of an incoming flow file to be deleted before transferring the flow file to the failure relationship. Also the lineage chain is overly complicated, as a new flow file is created just for transfer, while the original is dropped. This creates extra provenance events which can unnecessarily complicate the lineage.

> Add support for large result sets and normalizing Avro names to SelectHiveQL
> ----------------------------------------------------------------------------
>
>                 Key: NIFI-4473
>                 URL: https://issues.apache.org/jira/browse/NIFI-4473
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>            Priority: Major
>             Fix For: 1.5.0
>
>
> A number of enhancements were made to processors like QueryDatabaseTable to allow for such things as:
> - Splitting result sets into multiple flow files (i.e. Max Rows Per Flowfile property)
> - Max number of splits/rows returned (Max fragments)
> - Normalizing names to be Avro-compatible
> The RDBMS processors also now support Avro logical types, but the version of Avro needed by the current version of Hive (1.2.1) is Avro 1.7.7, which does not support logical types.
> These enhancements were made to JdbcCommon, but not to HiveJdbcCommon (the Hive version of the JDBC utils class). Since Hive queries can return even larger result sets than traditional RDBMS, these properties/enhancements are at least as valuable to have for SelectHiveQL.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)