You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "John Sichi (JIRA)" <ji...@apache.org> on 2010/01/20 22:53:54 UTC
[jira] Created: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Key: HIVE-1068
URL: https://issues.apache.org/jira/browse/HIVE-1068
Project: Hadoop Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: John Sichi
Fix For: 0.6.0
Zheng's description:
5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (HIVE-1068) CREATE VIEW followup: add a
"table type" enum attribute in metastore's MTable, and also null out
irrelevant attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-1068 started by John Sichi.
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805680#action_12805680 ]
John Sichi commented on HIVE-1068:
----------------------------------
I found out that the old version of thrift we are using does not support generation of native Java5 enums. Instead, it generates an obsolete type-unsafe enum pattern which will go away when we eventually upgrade thrift.
So, I'm thinking of just using a String in both thrift and JDO for the table type attribute, and then adding a native Java5 enum to wrap it on levels above thrift.
This attribute will be null in the persistence for existing metastore instances after upgrade, but I can infer the table type from other attributes (for external tables, we have a EXTERNAL=TRUE property set) and replace the null as part of retrieving the object definition.
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1068:
-----------------------------
Attachment: HIVE-1068.1.patch
Notes for reviewers:
(1)
I introduced the new TableType enum in package o.a.h.h.metastore. I didn't put it in metastore.api since it is not a generated file, and I didn't put it in metastore.model since that isn't accessed by most of Hive. If there's a better place to put it, please let me know.
For the enum values, I used MANAGED_TABLE, EXTERNAL_TABLE, and VIRTUAL_VIEW. This is so that later if we add MATERIALIZED_VIEW, there won't be any confusion. If we eventually use this for implementing the TABLE_TYPE attribute in the ResultSet for JDBC's getTables, we can use a CASE expression to fold these into simpler names (e.g. JDBC uses "TABLE" and "VIEW").
(2)
I tried making the SerDe itself null, but that caused problems in too many places that were requiring it to be non-null. So instead I nulled/emptied all of its attributes.
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806119#action_12806119 ]
John Sichi commented on HIVE-1068:
----------------------------------
See the updated output for create_view.q.out for an example of what we'll be storing in the view descriptor after this change.
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1068:
---------------------------------
Affects Version/s: (was: 0.6.0)
Release Note: (was: HIVE-1068. CREATE VIEW followup: add a 'table type' enum attribute in metastore's MTable. (John Sichi via zshao)
)
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch, HIVE-1068.2.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao resolved HIVE-1068.
------------------------------
Resolution: Fixed
Release Note:
HIVE-1068. CREATE VIEW followup: add a 'table type' enum attribute in metastore's MTable. (John Sichi via zshao)
Hadoop Flags: [Reviewed]
Committed. Thanks John!
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch, HIVE-1068.2.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806243#action_12806243 ]
Zheng Shao commented on HIVE-1068:
----------------------------------
The meaning of isNativeSerDe(String) changed. Can we rename it to "shouldGetColsFromSerDe" (and invert the condition)?
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1068:
-----------------------------
Attachment: HIVE-1068.2.patch
.2 patch addresses Zheng's comment and resolves a conflict with trunk.
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch, HIVE-1068.2.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1068) CREATE VIEW followup: add a "table
type" enum attribute in metastore's MTable, and also null out irrelevant
attributes for MTable instances which describe views
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806154#action_12806154 ]
John Sichi commented on HIVE-1068:
----------------------------------
I have written up metastore upgrade instructions here:
http://wiki.apache.org/hadoop/Hive/ViewDev#Metastore_Upgrades
> CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1068
> URL: https://issues.apache.org/jira/browse/HIVE-1068
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1068.1.patch
>
>
> Zheng's description:
> 5. TODO: Metadata change: We store "view" definitions in the same metadata table that we store "table" definitions.
> Shall we add a field "table type" so we know whether it's a table, external table, view, or materialized view in the future.
> We should clean up the additional useless fields in "view" - the test output shows that we are storing some garbage information for views.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.