You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Ning Zhang (JIRA)" <ji...@apache.org> on 2011/08/09 19:29:27 UTC

[jira] [Commented] (HIVE-2360) create dynamic partition if and only if intermediate source has files

    [ https://issues.apache.org/jira/browse/HIVE-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081780#comment-13081780 ] 

Ning Zhang commented on HIVE-2360:
----------------------------------

Franklin, can you create a review request? That's much easier to comment. Two comments here: 

 1) in dynamic partition inserts, a subquery corresponding to a partition is created only if there is a row that should be inserted into that partition, which means the directory should not be empty. If the directory is empty, most likely there is a bug there and we should fix that bug instead. 
 2) the function Utilities.getFileStatusRecurse() is quite expensive. We should avoid using it and use cached results (it's called somewhere else before). 

> create dynamic partition if and only if intermediate source has files
> ---------------------------------------------------------------------
>
>                 Key: HIVE-2360
>                 URL: https://issues.apache.org/jira/browse/HIVE-2360
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Franklin Hu
>            Assignee: Franklin Hu
>            Priority: Minor
>             Fix For: 0.8.0
>
>         Attachments: hive-2360.1.patch
>
>
> There are some conditions under which a partition description is created due to insert overwriting a table using dynamic partitioning for partitions that that are empty (have no files).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira