You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2010/08/02 21:21:17 UTC
[jira] Created: (HIVE-1507) Supply DriverContext to Hooks
Supply DriverContext to Hooks
-----------------------------
Key: HIVE-1507
URL: https://issues.apache.org/jira/browse/HIVE-1507
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Joydeep Sen Sarma
The DriverContext and the Context (linked off the latter) created during query compilation have information that's invaluable to writing hooks. In particular, the Context object has a cache of pathname to file size mappings looked up via hdfs. i would like to get access to this cache (for both reading and writing) in order to write a hook that depends on query size (for the purpose of dispatching it to the right cluster).
It's unfortunate we don't have a generic context object for hooks (into which we can add more stuff as needed). This is forcing an unnecessary api enhancement (we should be able to maintain backwards compatibility using reflection though). I think going forward we should have a generic context object with Session and Query related data inside.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1507) Supply DriverContext to Hooks
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1507:
-----------------------------
Issue Type: Improvement (was: Bug)
> Supply DriverContext to Hooks
> -----------------------------
>
> Key: HIVE-1507
> URL: https://issues.apache.org/jira/browse/HIVE-1507
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
>
> The DriverContext and the Context (linked off the latter) created during query compilation have information that's invaluable to writing hooks. In particular, the Context object has a cache of pathname to file size mappings looked up via hdfs. i would like to get access to this cache (for both reading and writing) in order to write a hook that depends on query size (for the purpose of dispatching it to the right cluster).
> It's unfortunate we don't have a generic context object for hooks (into which we can add more stuff as needed). This is forcing an unnecessary api enhancement (we should be able to maintain backwards compatibility using reflection though). I think going forward we should have a generic context object with Session and Query related data inside.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1507) Supply DriverContext to Hooks
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894670#action_12894670 ]
John Sichi commented on HIVE-1507:
----------------------------------
Yes, regarding API enhancement, we ran into a similar situation with GenericUDAFResolver. We dealt with it by adding an Info class as extendable context, adding a new interface which passes in the info class, and then using reflection for continuing to support the old interface as well to avoid breaking it.
> Supply DriverContext to Hooks
> -----------------------------
>
> Key: HIVE-1507
> URL: https://issues.apache.org/jira/browse/HIVE-1507
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
>
> The DriverContext and the Context (linked off the latter) created during query compilation have information that's invaluable to writing hooks. In particular, the Context object has a cache of pathname to file size mappings looked up via hdfs. i would like to get access to this cache (for both reading and writing) in order to write a hook that depends on query size (for the purpose of dispatching it to the right cluster).
> It's unfortunate we don't have a generic context object for hooks (into which we can add more stuff as needed). This is forcing an unnecessary api enhancement (we should be able to maintain backwards compatibility using reflection though). I think going forward we should have a generic context object with Session and Query related data inside.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.