[jira] [Commented] (HIVE-3208) Multiformat tables and partitions

["Carl Steinbach (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-3208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402933#comment-13402933 ] 

Carl Steinbach commented on HIVE-3208:
--------------------------------------

The Hive Metastore currently forces clients to associate a specific file format with every table and partition. The goal of this feature from a user's standpoint is to make tables and partitions as close to format independent as possible. This means that Hive will need to be able to transparently map a single logical table or partition to data stored in heterogenous formats.

Motivating use cases:
* The ability to automatically convert table/partition data from one format to another format in the background without requiring the user to stream the data into a new table/partition.
* Simplifying import/export.
* Simplifying DDL for most commands.
* Simplifying ETL workflows.

Most of this functionality can already be achieved manually by adding an additional 'format' partition key to each table, which suggests adding 'hidden' or 'pseudo' partition keys may be one possible way of implementing this feature.

I'm working on a design document and will get that posted soon.
                
> Multiformat tables and partitions
> ---------------------------------
>
>                 Key: HIVE-3208
>                 URL: https://issues.apache.org/jira/browse/HIVE-3208
>             Project: Hive
>          Issue Type: New Feature
>          Components: Metastore
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira