You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2008/11/29 01:02:44 UTC
[jira] Created: (HIVE-86) drop table should not delete data for
external tables
drop table should not delete data for external tables
-----------------------------------------------------
Key: HIVE-86
URL: https://issues.apache.org/jira/browse/HIVE-86
Project: Hadoop Hive
Issue Type: Bug
Reporter: Joydeep Sen Sarma
Priority: Critical
as per title
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-86) drop table should not delete data for
external tables
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-86:
--------------------------------
Attachment: HIVE-86.patch
I've updated the patch to do all the checks server side as suggested, much cleaner.
The reason I didn't override the deleteData boolean in the previous patch is that I thought it might be useful if someone wanted to force delete an external table for whatever reason. It's probably better to be safe then sorry though.
I left in the changes in RWTable, but I assume those file based metastore will be removed eventually? It would help reduce confusion. Perhaps open a ticket for it?
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-86) drop table should not delete data for
external tables
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-86:
--------------------------------
Attachment: HIVE-86.patch
A first attempt at a patch for this issue. Please review and point me in the right direction. Since I haven't worked much with Hive I'm having trouble seeing how the different components work together. It seems like there are two places where the data in a dropped table can be deleted, once in RWTable.java and one in HiveMetaStoreClient.java. Any reason for this duplication?
What role does the different *Store objects in the metastore java dir have?
This patch does not make sure that a partition drop doesn't delete the original data. Shall I open up a new issue for that?
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-86) drop table should not delete data for
external tables
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Thusoo updated HIVE-86:
------------------------------
Component/s: Query Processor
Assigning component.
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-86) drop table should not delete data for
external tables
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao resolved HIVE-86.
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
HIVE-86. Drop table should not delete data for external tables.
(Johan Oskarsson through zshao)
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-86) drop table should not delete data for
external tables
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652660#action_12652660 ]
Joydeep Sen Sarma commented on HIVE-86:
---------------------------------------
thx! Patch looks pretty good.
Couple of concerns:
- i think we can just have one version of the isExternal command. The ql/.../Table.java has a metastore.api.Table object - and we can have an isExternal definition over that which can be shared in both places.
- other concern is that the isExternal setting is being checked by the client side code. it seems to me that the server side should be enforcing this setting (otherwise any thrift client can come and delete data based on the thrift api regardless of the external setting). So we should (as a safety measure) be enforcing this check in HiveMetaStore.java:drop_table as well (regardless of the delete_data setting.)
regarding RWTable versus HiveMetaStoreClient - sorry for all the confusion. The thing is that there are two implementations of the MetaStore available - one db backed which is the new one that we use normally. There is an old one that's file based and RWTable is used by that one (AFAIK).
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-86) drop table should not delete data for
external tables
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653000#action_12653000 ]
Joydeep Sen Sarma commented on HIVE-86:
---------------------------------------
+1. looks good.
Ashish or Zheng - can you please commit this?
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-86) drop table should not delete data for
external tables
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-86:
-------------------------------
Fix Version/s: 0.3.0
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Fix For: 0.3.0
>
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-86) drop table should not delete data for
external tables
Posted by "David Phillips (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Phillips updated HIVE-86:
-------------------------------
Description: We should not delete data for external tables. (was: as per title)
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Priority: Critical
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-86) drop table should not delete data for
external tables
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653042#action_12653042 ]
Zheng Shao commented on HIVE-86:
--------------------------------
Committed. svn revision 723124.
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-86) drop table should not delete data for
external tables
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653013#action_12653013 ]
Zheng Shao commented on HIVE-86:
--------------------------------
Applied the patch. Rerunning the test now. Will commit both internally and in apache.
> drop table should not delete data for external tables
> -----------------------------------------------------
>
> Key: HIVE-86
> URL: https://issues.apache.org/jira/browse/HIVE-86
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Priority: Critical
> Attachments: HIVE-86.patch, HIVE-86.patch
>
>
> We should not delete data for external tables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.