You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Hao Liu (JIRA)" <ji...@apache.org> on 2008/12/09 08:17:44 UTC
[jira] Created: (HIVE-141) drop table partition behaving oddly -
does not create subdirectories
drop table partition behaving oddly - does not create subdirectories
--------------------------------------------------------------------
Key: HIVE-141
URL: https://issues.apache.org/jira/browse/HIVE-141
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.20.0
Reporter: Hao Liu
for example, I have a table, which has two partitions:
tmp_table_name/dt=2008-11-01
tmp_table_name/dt=2008-11-02
When we use hive metastore to drop the first partition (as root), I expect the data file will be moved to user/root/.Trash/081103xxxx/tmp_table_name/dt=2008-11-01 by default. However, directory tmp_table_name was not created, the data was moved to user/root/.Trash/081103xxxx/dt=2008-11-01, which makes data recovery a very difficult task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-141) drop table partition behaving oddly -
does not create subdirectories
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasad Chakka resolved HIVE-141.
--------------------------------
Resolution: Fixed
fixed by patching hadoop
> drop table partition behaving oddly - does not create subdirectories
> --------------------------------------------------------------------
>
> Key: HIVE-141
> URL: https://issues.apache.org/jira/browse/HIVE-141
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Hao Liu
> Assignee: Prasad Chakka
> Priority: Critical
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> for example, I have a table, which has two partitions:
> tmp_table_name/dt=2008-11-01
> tmp_table_name/dt=2008-11-02
> When we use hive metastore to drop the first partition (as root), I expect the data file will be moved to user/root/.Trash/081103xxxx/tmp_table_name/dt=2008-11-01 by default. However, directory tmp_table_name was not created, the data was moved to user/root/.Trash/081103xxxx/dt=2008-11-01, which makes data recovery a very difficult task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-141) drop table partition behaving oddly -
does not create subdirectories
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasad Chakka reassigned HIVE-141:
----------------------------------
Assignee: Prasad Chakka
> drop table partition behaving oddly - does not create subdirectories
> --------------------------------------------------------------------
>
> Key: HIVE-141
> URL: https://issues.apache.org/jira/browse/HIVE-141
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Hao Liu
> Assignee: Prasad Chakka
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> for example, I have a table, which has two partitions:
> tmp_table_name/dt=2008-11-01
> tmp_table_name/dt=2008-11-02
> When we use hive metastore to drop the first partition (as root), I expect the data file will be moved to user/root/.Trash/081103xxxx/tmp_table_name/dt=2008-11-01 by default. However, directory tmp_table_name was not created, the data was moved to user/root/.Trash/081103xxxx/dt=2008-11-01, which makes data recovery a very difficult task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-141) drop table partition behaving oddly -
does not create subdirectories
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659000#action_12659000 ]
Prasad Chakka commented on HIVE-141:
------------------------------------
this seems to be result of a bug/feature in Hadoop Trash.moveToTrash() function
The comment for the class Trash (see below) says that the deleted directory should retail the full path in .Trash/Current directory but the actual code is only the name of the directory is used. I will try to check the actual policy of the deletes and see whether we need to implement our own moveToTrash function.
/** Provides a <i>trash</i> feature. Files are moved to a user's trash
* directory, a subdirectory of their home directory named ".Trash". Files are
* initially moved to a <i>current</i> sub-directory of the trash directory.
* Within that sub-directory their original path is preserved. Periodically
* one may checkpoint the current trash and remove older checkpoints. (This
* design permits trash management without enumeration of the full trash
* content, without date support in the filesystem, and without clock
* synchronization.)
*/
> drop table partition behaving oddly - does not create subdirectories
> --------------------------------------------------------------------
>
> Key: HIVE-141
> URL: https://issues.apache.org/jira/browse/HIVE-141
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Hao Liu
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> for example, I have a table, which has two partitions:
> tmp_table_name/dt=2008-11-01
> tmp_table_name/dt=2008-11-02
> When we use hive metastore to drop the first partition (as root), I expect the data file will be moved to user/root/.Trash/081103xxxx/tmp_table_name/dt=2008-11-01 by default. However, directory tmp_table_name was not created, the data was moved to user/root/.Trash/081103xxxx/dt=2008-11-01, which makes data recovery a very difficult task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-141) drop table partition behaving oddly -
does not create subdirectories
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Thusoo updated HIVE-141:
-------------------------------
Priority: Critical (was: Major)
need resolution on this before the first release.
> drop table partition behaving oddly - does not create subdirectories
> --------------------------------------------------------------------
>
> Key: HIVE-141
> URL: https://issues.apache.org/jira/browse/HIVE-141
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Hao Liu
> Assignee: Prasad Chakka
> Priority: Critical
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> for example, I have a table, which has two partitions:
> tmp_table_name/dt=2008-11-01
> tmp_table_name/dt=2008-11-02
> When we use hive metastore to drop the first partition (as root), I expect the data file will be moved to user/root/.Trash/081103xxxx/tmp_table_name/dt=2008-11-01 by default. However, directory tmp_table_name was not created, the data was moved to user/root/.Trash/081103xxxx/dt=2008-11-01, which makes data recovery a very difficult task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.