[jira] [Commented] (HIVE-15571) Support Insert into for druid storage handler

["Nishant Bangarwa (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033510#comment-16033510 ] 

Nishant Bangarwa commented on HIVE-15571:
-----------------------------------------

[~jcamachorodriguez] handled review comments and updated JIRA desc. 

> Support Insert into for druid storage handler
> ---------------------------------------------
>
>                 Key: HIVE-15571
>                 URL: https://issues.apache.org/jira/browse/HIVE-15571
>             Project: Hive
>          Issue Type: New Feature
>          Components: Druid integration
>            Reporter: slim bouguerra
>            Assignee: Nishant Bangarwa
>         Attachments: HIVE-15571.01.patch, HIVE-15571.02.patch
>
>
> Add support of inset into operator for druid storage handler.
> Implementation - 
> 1) Using DruidOutputFormat, multiple reducers write segments to a staging directory with the new versions. 
> 2) When doing insert into, before publishing the segments to the metadata storage, versions for the segments are updated based on the existing segments in druid metadata storage. 
> 3) After final versions are checked, the segments are moved from the staging directory to the segment directory. 
> 4) New segment versions are published to the metadata storage. 
> Reading of existing segments from metadata storage and updating the versions of newly created segments based on them is done in a single transaction. This will make sure that segments generated by parallel Insert Into statements being executed will not overwrite each other. In case of conflict in metadata storage transaction insert into will fail.   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)