You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "John Sichi (JIRA)" <ji...@apache.org> on 2010/01/21 00:26:54 UTC
[jira] Created: (HIVE-1079) CREATE VIEW followup: derive
dependencies on underlying base table partitions from view definition
CREATE VIEW followup: derive dependencies on underlying base table partitions from view definition
---------------------------------------------------------------------------------------------------
Key: HIVE-1079
URL: https://issues.apache.org/jira/browse/HIVE-1079
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: John Sichi
Fix For: 0.6.0
When querying a view, it would be useful to know which underlying base table partitions it depends on in order to know how fresh the result is (or to be able to wait until all of those partitions have been loaded consistently). The task is to come up with a way to perform this analysis automatically (possibly overconservatively), or alternately to let the view creator annotate the view definition with this dependency information, or some combination of the two.
Note that this would be useful for any complex query which directly accesses base tables (not just view definitions).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1079) CREATE VIEW followup: derive
dependencies on underlying base table partitions from view definition
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879169#action_12879169 ]
John Sichi commented on HIVE-1079:
----------------------------------
For the manual approach, a very simple thing we can start with is just letting users add partitions to views explicitly as a way of indicating that the underlying data is ready.
> CREATE VIEW followup: derive dependencies on underlying base table partitions from view definition
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1079
> URL: https://issues.apache.org/jira/browse/HIVE-1079
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
>
> When querying a view, it would be useful to know which underlying base table partitions it depends on in order to know how fresh the result is (or to be able to wait until all of those partitions have been loaded consistently). The task is to come up with a way to perform this analysis automatically (possibly overconservatively), or alternately to let the view creator annotate the view definition with this dependency information, or some combination of the two.
> Note that this would be useful for any complex query which directly accesses base tables (not just view definitions).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1079) CREATE VIEW followup: derive
dependencies on underlying base table partitions from view definition
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831698#action_12831698 ]
John Sichi commented on HIVE-1079:
----------------------------------
Additional user feedback:
(1) Take the 80/20 approach and focus on the single-table use case for starters; don't get hung up on the complicated join/union cases.
(2) Propagating metadata about the partitioning key through to the view will be important in UI's which auto-suggest a filter on the partitioning columns. A nuance here is that if the filter is baked into the view definition itself (i.e. the view is partition-specific), then we might not want to propagate it (if the view creator forgot to project it away).
(3) It would be nice if a system function CURRENT_PARTITION() were provided so that the view definition could bake in a dynamic filter like WHERE ds=CURRENT_PARTITION(). This is an independent but related feature whose semantics remain to be worked out.
> CREATE VIEW followup: derive dependencies on underlying base table partitions from view definition
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1079
> URL: https://issues.apache.org/jira/browse/HIVE-1079
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.6.0
>
>
> When querying a view, it would be useful to know which underlying base table partitions it depends on in order to know how fresh the result is (or to be able to wait until all of those partitions have been loaded consistently). The task is to come up with a way to perform this analysis automatically (possibly overconservatively), or alternately to let the view creator annotate the view definition with this dependency information, or some combination of the two.
> Note that this would be useful for any complex query which directly accesses base tables (not just view definitions).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1079) CREATE VIEW followup: derive
dependencies on underlying base table partitions from view definition
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach reassigned HIVE-1079:
------------------------------------
Assignee: Carl Steinbach (was: John Sichi)
> CREATE VIEW followup: derive dependencies on underlying base table partitions from view definition
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1079
> URL: https://issues.apache.org/jira/browse/HIVE-1079
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: Carl Steinbach
>
> When querying a view, it would be useful to know which underlying base table partitions it depends on in order to know how fresh the result is (or to be able to wait until all of those partitions have been loaded consistently). The task is to come up with a way to perform this analysis automatically (possibly overconservatively), or alternately to let the view creator annotate the view definition with this dependency information, or some combination of the two.
> Note that this would be useful for any complex query which directly accesses base tables (not just view definitions).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.