You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "He Yongqiang (JIRA)" <ji...@apache.org> on 2010/11/04 21:28:42 UTC
[jira] Created: (HIVE-1767) Merge files does not work with dynamic
partition
Merge files does not work with dynamic partition
------------------------------------------------
Key: HIVE-1767
URL: https://issues.apache.org/jira/browse/HIVE-1767
Project: Hive
Issue Type: Bug
Reporter: He Yongqiang
Assignee: He Yongqiang
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1767) Merge files does not work with
dynamic partition
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928361#action_12928361 ]
He Yongqiang commented on HIVE-1767:
------------------------------------
When there is a dynamic partition, the merge task never gets launched. I was trying to figure out what's happening there.
It turns out a bug in ConditionalResolverMergeFiles that leads to the size information counted is not correct for dp queries.
Attached a patch for it.
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1767) Merge files does not work with
dynamic partition
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928394#action_12928394 ]
Namit Jain commented on HIVE-1767:
----------------------------------
+1
running tests
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1767) Merge files does not work with
dynamic partition
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928382#action_12928382 ]
Namit Jain commented on HIVE-1767:
----------------------------------
load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket20.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket21.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket22.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket23.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
The test needs to be updated
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1767:
-------------------------------
Attachment: HIVE-1767.1.patch
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1767:
-------------------------------
Attachment: HIVE-1767.1.patch
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1767) Merge files does not work with
dynamic partition
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928344#action_12928344 ]
Ning Zhang commented on HIVE-1767:
----------------------------------
Can you elaborate the bug in the description?
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-1767.
------------------------------
Resolution: Fixed
Fix Version/s: 0.7.0
Hadoop Flags: [Reviewed]
Committed. Thanks Yongqiang
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Fix For: 0.7.0
>
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1767:
-------------------------------
Status: Patch Available (was: Open)
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1767:
-----------------------------
Status: Open (was: Patch Available)
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-1767.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1767) Merge files does not work with dynamic
partition
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1767:
-------------------------------
Attachment: (was: HIVE-1767.1.patch)
> Merge files does not work with dynamic partition
> ------------------------------------------------
>
> Key: HIVE-1767
> URL: https://issues.apache.org/jira/browse/HIVE-1767
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.