[jira] [Commented] (NIFI-5517) PutHive3Streaming does not correctly support all Hive types

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580277#comment-16580277 ] 

ASF GitHub Bot commented on NIFI-5517:
--------------------------------------

GitHub user mattyb149 opened a pull request:

    https://github.com/apache/nifi/pull/2950

    NIFI-5517: Add support for remaining Hive types to PutHive3Streaming

    Thank you for submitting a contribution to Apache NiFi.
    
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    
    - [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    
    - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    - [x] Is your initial contribution a single, squashed commit?
    
    ### For code changes:
    - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
    - [x] Have you written or updated unit tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? 
    - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
    - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
    - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
    
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
    
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mattyb149/nifi NIFI-5517

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/2950.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2950
    
----
commit 5118be13b0926e39e4ecd4004c18297c36a35fef
Author: Matthew Burgess <ma...@...>
Date:   2018-08-14T18:06:46Z

    NIFI-5517: Add support for remaining Hive types to PutHive3Streaming

----


> PutHive3Streaming does not correctly support all Hive types
> -----------------------------------------------------------
>
>                 Key: NIFI-5517
>                 URL: https://issues.apache.org/jira/browse/NIFI-5517
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>            Priority: Major
>
> NIFI-5475 upgraded the version of Hive 3 to Apache Hive 3.1.0, and some code changes had to be made as there were new Writable types to be used in place of the old ones. As a result of the upgrade, NIFI-5491 was discovered and included some fixes for PutHive3Streaming to support various primitive data types as well as structs.
> However it appears that NIFI-5491 did not cover all the available Hive types, and NIFI-5475's changes were to PutORC for time/date types, but similar changes have been made to the Hive writer as well.
> PutHive3Streaming should support all available Hive types where prudent (i.e. where NiFi Record Field Types can be converted to Hive Column Types). The current list of "top-level" types include:
> PRIMITIVE, LIST, MAP, STRUCT, UNION
> And the current list of PRIMITIVE types include:
> VOID, BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, STRING,
>     DATE, TIMESTAMP, TIMESTAMPLOCALTZ, BINARY, DECIMAL, VARCHAR, CHAR,
>     INTERVAL_YEAR_MONTH, INTERVAL_DAY_TIME, UNKNOWN
> As of NIFI-5475 I believe PutHive3Streaming supports BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, STRING, VARCHAR, and CHAR for primitive types, and with respect to the supported primitive types, PutHive3Streaming supports LIST, MAP, STRUCT, and UNION.
> The remaining list of primitive types is DATE, TIMESTAMP, BINARY, and DECIMAL. VOID and UNKNOWN are used "under the hood" I believe, and ORC doesn't currently support TIMESTAMPLOCALTZ. INTERVAL_YEAR_MONTH and INTERVAL_DAY_TIME are only valid for queries and not as column types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)