You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Keegan Mosley (JIRA)" <ji...@apache.org> on 2012/07/05 20:02:35 UTC
[jira] [Created] (HIVE-3231) msck repair should find partitions
already containing data files
Keegan Mosley created HIVE-3231:
-----------------------------------
Summary: msck repair should find partitions already containing data files
Key: HIVE-3231
URL: https://issues.apache.org/jira/browse/HIVE-3231
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0, 0.9.1
Reporter: Keegan Mosley
Priority: Minor
msck repair currently will only discover partition directories if they are empty.
It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keegan Mosley updated HIVE-3231:
--------------------------------
Priority: Major (was: Minor)
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keegan Mosley updated HIVE-3231:
--------------------------------
Fix Version/s: 0.10.0
Status: Patch Available (was: Open)
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Priority: Minor
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3231:
---------------------------------
Status: Open (was: Patch Available)
@Keegan: This patch needs to be rebased on trunk. Also, repair.q has been split into repair.q and repair_hadoop23.q, so both files probably need to be updated. Finally, would you mind submitting a review request for this on either phabricator or reviewboard? Thanks.
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HIVE-3231) msck repair should find
partitions already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-3231 started by Keegan Mosley.
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Assignee: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt, HIVE-3231.2.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keegan Mosley updated HIVE-3231:
--------------------------------
Assignee: Carl Steinbach (was: Keegan Mosley)
Status: Patch Available (was: In Progress)
https://reviews.apache.org/r/7649/
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Assignee: Carl Steinbach
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt, HIVE-3231.2.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3231:
---------------------------------
Component/s: Metastore
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keegan Mosley updated HIVE-3231:
--------------------------------
Attachment: HIVE-3231.1.patch.txt
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Priority: Minor
> Labels: msck
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3231:
---------------------------------
Assignee: Keegan Mosley
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Assignee: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3231) msck repair should find partitions
already containing data files
Posted by "Keegan Mosley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keegan Mosley updated HIVE-3231:
--------------------------------
Attachment: HIVE-3231.2.patch.txt
> msck repair should find partitions already containing data files
> ----------------------------------------------------------------
>
> Key: HIVE-3231
> URL: https://issues.apache.org/jira/browse/HIVE-3231
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.10.0, 0.9.1
> Reporter: Keegan Mosley
> Assignee: Keegan Mosley
> Labels: msck
> Fix For: 0.10.0
>
> Attachments: HIVE-3231.1.patch.txt, HIVE-3231.2.patch.txt
>
>
> msck repair currently will only discover partition directories if they are empty.
> It seems a more apt use case to copy data files into a table, creating the partition directories as you go, rather than creating a bunch of empty partition directories, then running msck repair to dynamically add them, then inserting your actual data files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira