[jira] [Commented] (HIVE-6405) Support append feature for HCatalog

["Sushanth Sowmyan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906058#comment-13906058 ] 

Sushanth Sowmyan commented on HIVE-6405:
----------------------------------------

After a brief discussion with Ashutosh on the nature of HIVE-6406, I'm going to create another task to add ql grammar to support modification of the immutability property in a manner similar to existing grammar for NO_DROP/OFFLINE, so that this can be treated as another kind of data protection, and so that users will not have to deal with explicitly modifying TBLPROPERTIES.

> Support append feature for HCatalog
> -----------------------------------
>
>                 Key: HIVE-6405
>                 URL: https://issues.apache.org/jira/browse/HIVE-6405
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Metastore, Query Processor, Thrift API
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-6405.patch
>
>
> HCatalog currently treats all tables as "immutable" - i.e. all tables and partitions can be written to only once, and not appended. The nuances of what this means is as follows:
>  * A non-partitioned table can be written to, and data in it is never updated from then on unless you drop and recreate.
>  * A partitioned table may support "appending" of a sort in a manner by adding new partitions to the table, but once written, the partitions themselves cannot have any new data added to them.
> Hive, on the other hand, does allow us to "INSERT INTO" into a table, thus allowing us append semantics. There is benefit to both of these models, and so, our goal is as follows:
> a) Introduce a notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow "INSERT INTO" into it from hive. This property being set will allow hive to mimic HCatalog's current immutable-table property. (I'm going to create a separate sub-task to cover this bit, and focus on the HCatalog-side on this jira)
> b) As long as that flag is not set, HCatalog should be changed to allow appends into it as well, and not simply error out if data already exists in a table.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)