[jira] Commented: (HIVE-1602) List Partitioning

["Joydeep Sen Sarma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903606#action_12903606 ] 

Joydeep Sen Sarma commented on HIVE-1602:
-----------------------------------------

hmmm - not sure i understand. how can we collapse partitions? we have to generate one directory per distinct DP column value - no?

(or are you thinking of jumping straight to har?)

> List Partitioning
> -----------------
>
>                 Key: HIVE-1602
>                 URL: https://issues.apache.org/jira/browse/HIVE-1602
>             Project: Hadoop Hive
>          Issue Type: New Feature
>    Affects Versions: 0.7.0
>            Reporter: Ning Zhang
>
> Dynamic partition inserts create partitions bases on the dynamic partition column values. Currently it creates one partition for each distinct DP column value. This could result in skews in the created dynamic partitions in that some partitions are large but there could be large number of small partitions as well. This results in burdens in HDFS as well as metastore. A list partitioning scheme that aggregate a number of small partitions into one big one is more preferable for skewed partitions. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.