[jira] [Commented] (FLUME-1734) Create a HCatalog Sink

["Roshan Naik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604775#comment-13604775 ] 

Roshan Naik commented on FLUME-1734:
------------------------------------

FYI: Due to limitations in the current Hive and HCatalog support for streaming clients, a new initiative is being pursued in Hive (and HCatalog) to support streaming clients.

https://issues.apache.org/jira/browse/HIVE-4196

One of things being baked into it the ability to support transactional commits needed by Flume sinks.
                
> Create a HCatalog Sink 
> -----------------------
>
>                 Key: FLUME-1734
>                 URL: https://issues.apache.org/jira/browse/FLUME-1734
>             Project: Flume
>          Issue Type: New Feature
>          Components: Sinks+Sources
>    Affects Versions: v1.2.0
>            Reporter: Roshan Naik
>            Assignee: Roshan Naik
>              Labels: features
>
> Create a sink that would stream data into HCatalog partitions. The primary goal being that once the data is loaded into Hadoop, it should be automatically queryable (using say Hive or Pig) without requiring additional post processing steps on behalf of the users. Sink should manage the creation of new partitions and committing them periodically. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira