You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@nifi.apache.org by "Matt Burgess (JIRA)" <ji...@apache.org> on 2016/05/11 20:37:12 UTC

[jira] [Created] (NIFI-1868) Add support for Hive Streaming

Matt Burgess created NIFI-1868:
----------------------------------

             Summary: Add support for Hive Streaming
                 Key: NIFI-1868
                 URL: https://issues.apache.org/jira/browse/NIFI-1868
             Project: Apache NiFi
          Issue Type: New Feature
            Reporter: Matt Burgess
            Assignee: Matt Burgess
             Fix For: 1.0.0, 0.7.0


Traditionally adding new data into Hive requires gathering a large amount of data onto HDFS and then periodically adding a new partition. This is essentially a “batch insertion”. Insertion of new data into an existing partition is not permitted. Hive Streaming API allows data to be pumped continuously into Hive. The incoming data can be continuously committed in small batches of records into an existing Hive partition or table. Once data is committed it becomes immediately visible to all Hive queries initiated subsequently.

This case is to add a PutHiveStreaming processor to NiFi, to leverage the Hive Streaming API to allow continuous streaming of data into a Hive partition/table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)