You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/04/14 19:45:41 UTC
[jira] [Commented] (TRAFODION-2584) Add support to register hive
objects in trafodion metadata
[ https://issues.apache.org/jira/browse/TRAFODION-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969450#comment-15969450 ]
ASF GitHub Bot commented on TRAFODION-2584:
-------------------------------------------
GitHub user anoopsharma00 opened a pull request:
https://github.com/apache/incubator-trafodion/pull/1066
[TRAFODION-2584] Add support to register hive objects in traf metadata
Syntax:
register hive table/view [if not exists] <object-name> [cascade]
unregister hive table/view [if exists] <object-name> [cascade]
cascade option: register/unregister all underlying objects that are
part of the specified view
update statistics, grant/revoke, traf views or external table creation
on hive objects will automatically and internally register those objects
in trafodion metadata.
invoke/showddl will show if this object is registered and whether
that registration was internal or explicit.
Get command extensions:
get hive registered tables/view/objects in catalog trafodion;
get hive external tables in catalog trafodion;
Cleanup command extensions:
cleanup metadata command will clean up inconsistent hive objects
(underlying hive object is missing but object is registered or
an external table exists)
cleanup [hive table | hive view] on <object-name>;
Existing hive objects that had implicit or explicit external tables
created prior to this checkin will have no change in behavior.
ObjectUID of those external tables will continue to be used for
relevant operations.
One can drop those external tables and explicitly register them,
or a subsequent operation(upd stats, grant, etc) that needs objectUID will
automatically register them.
minor changes based on review comments of previous checkin
get all objects command on hive metadata no longer fails.
get views on objects return 3-part name that could be used to
differentiate between a traf and hive view.
regress/hive/TEST007 has been extended.
TBD: Add register/unregister privileges
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/anoopsharma00/incubator-trafodion ansharma_hivereg_br_deliver
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/1066.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1066
----
commit 9c58649f930a799f03a2703d012371e4e66fa1a2
Author: Anoop Sharma <an...@esgyn.com>
Date: 2017-04-14T19:38:18Z
[TRAFODION-2584] Add support to register hive objects in traf metadata
Syntax:
register hive table/view [if not exists] <object-name> [cascade]
unregister hive table/view [if exists] <object-name> [cascade]
cascade option: register/unregister all underlying objects that are
part of the specified view
update statistics, grant/revoke, traf views or external table creation
on hive objects will automatically and internally register those objects
in trafodion metadata.
invoke/showddl will show if this object is registered and whether
that registration was internal or explicit.
Get command extensions:
get hive registered tables/view/objects in catalog trafodion;
get hive external tables in catalog trafodion;
Cleanup command extensions:
cleanup metadata command will clean up inconsistent hive objects
(underlying hive object is missing but object is registered or
an external table exists)
cleanup [hive table | hive view] on <object-name>;
Existing hive objects that had implicit or explicit external tables
created prior to this checkin will have no change in behavior.
ObjectUID of those external tables will continue to be used for
relevant operations.
One can drop those external tables and explicitly register them,
or a subsequent operation(upd stats, grant, etc) that needs objectUID will
automatically register them.
minor changes based on review comments of previous checkin
get all objects command on hive metadata no longer fails.
get views on objects return 3-part name that could be used to
differentiate between a traf and hive view.
regress/hive/TEST007 has been extended.
TBD: Add register/unregister privileges
----
> Add support to register hive objects in trafodion metadata
> ----------------------------------------------------------
>
> Key: TRAFODION-2584
> URL: https://issues.apache.org/jira/browse/TRAFODION-2584
> Project: Apache Trafodion
> Issue Type: Improvement
> Reporter: Anoop Sharma
> Assignee: Anoop Sharma
>
> Many trafodion operations on hive objects require an objectUID (a unique
> identifier) to be associated with them. This allows them to be added to
> trafodion metadata and then referred to later.
> Operations that need an identifier are privileges grant/revoke,
> updating statistics, hive view usage and external tables.
> Currently, to associate an objectUID with a hive table, trafodion creates
> an implicit external table on it. That generates a UID for the table
> which is then associated with the underlying object.
> That approach is not lightweight in that the implicit external table
> creation impact multiple traf metadata tables as it goes through all the
> steps needed for table creation.
> It doesn't support hive views as traf doesnt allow creation of external
> tables on a hive view.
> Also, If an implicit external table has been created by a traf operation,
> for example update statistics, and then the user wants to create an
> explicit external table, they need to drop the implicit table before
> creating the new external table. It means that all stats become
> obsolete as they are associated with the older implicit external
> table objectUID.
> This enhancement proposes a way to register hive objects(tables, views)
> in trafodion metadata OBJECTS without creating an external table.
> That has the following advantages:
> -- registering objects is a lightweight operation without creating an
> external table
> -- it impact only OBJECTS metadata and doesnt affect other objects
> (like COLUMNS, KEYS, etc) which will be for an external table
> -- it can be used for hive tables and views
> -- since there is no implicit external table, this allows creation of user
> external tables without having to drop the implicit table and thus
> invalidating the older objectUID. User external table create/drop
> become independent of internal operations that require an objectUID.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)