You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/11/12 01:32:14 UTC
[jira] Created: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
change Pre/Post Query Hooks to take in 1 parameter: HookContext
---------------------------------------------------------------
Key: HIVE-1785
URL: https://issues.apache.org/jira/browse/HIVE-1785
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Namit Jain
Assignee: Liyin Tang
This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive-1785_3.patch
In order to be compatible, we check whether the hook implements the interface, which runs with the hook context.
If not, just call the originally interface.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1785:
-----------------------------
Attachment: (was: hive_1758_5.patch)
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive-1785_6.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive_1758_5.patch
1) make the old interface be deprecated
2) let the existing Prehook and Posthook implements the new interface.
3) the task tag for each task
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1758_5.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive_1785_1.patch
In this patch I have changed the interface of pre hoook and post hook.
So all the hooks will take the HookContext as parameter.
In HookContext, it has the QueryPlan, HiveConf and a list of Completed Tasks.
It will be easier to extend HookContext in the future if more information the hook needs.
By the way, I generate diff based on Hive-1642( converting join into map join), assuming this patch will be committed after Hive-1642.
Please review :)
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive_1785_1.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931291#action_12931291 ]
Namit Jain commented on HIVE-1785:
----------------------------------
That's a good idea - we can change our internal hooks slowly
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933230#action_12933230 ]
John Sichi commented on HIVE-1785:
----------------------------------
+1.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive-1785_4.patch
In this patch, I add the Hook interface over Pre/PostExecute and ExecuteWithHookContext interface.
In the future, user can only implements ExecuteWithHookContext instead of Pre/PostExecute.
Also it is compatible with old hooks.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Status: Patch Available (was: Open)
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1758_5.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1785:
-----------------------------
Status: Open (was: Patch Available)
Discussed with Liyin, and there are a few more things we should do here:
* add Javadoc for Hook, HookContext, and ExecuteWithHookContext
* add deprecation annotation for old PreExecute/PostExecute
* rewrite existing hook implementations in Hive codebase to implement new interface
* run Facebook-specific tests to validate backwards-compatibility during the transition
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931287#action_12931287 ]
John Sichi commented on HIVE-1785:
----------------------------------
Can we use reflection to make this change backwards-compatible?
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932623#action_12932623 ]
Liyin Tang commented on HIVE-1785:
----------------------------------
I generate the diff based on the Hive-1642.
Please ignore the irrelevant code and output file.
Sorry for the inconvenient.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933096#action_12933096 ]
Liyin Tang commented on HIVE-1785:
----------------------------------
How about adding one more layer over Pre/PostExecute interface, call it Hook.
So both ExecuteWithHookContext and Pre/PostExecute implements this Hook interface
During the run time, using reflection to see whether the hook is ExecuteWithHookContext or Pre/PostExecute.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi resolved HIVE-1785.
------------------------------
Resolution: Fixed
Release Note: PreExecute and PostExecute have been deprecated in favor of ExecuteWithHookContext.
Committed. Thanks Liyin!
Could you explain this change on the user mailing list? Also, we need a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml (I just remembered that).
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive-1785_6.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1785:
-----------------------------
Hadoop Flags: (was: [Incompatible change])
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive_1785_2.patch
Thanks for John's comments.
Now in the Driver.java:
for (PostExecute peh : getPostExecHooks()) {
if (peh instanceof ExecuteWithHookContext) {
((ExecuteWithHookContext) peh).run(hookContext);
} else {
peh.run(SessionState.get(), plan.getInputs(), plan.getOutputs(),
(SessionState.get() != null ? SessionState.get().getLineageState().getLineageInfo()
: null), ShimLoader.getHadoopShims().getUGIForConf(conf));
}
}
Let's discuss about the interface of HookContext. How much information we should keep in the HookContext.
Right now, I keep the query plan, job conf and completed tasks.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1785:
-----------------------------
Fix Version/s: 0.7.0
Affects Version/s: 0.7.0
Status: Open (was: Patch Available)
* It looks like you mixed in some changes from other patches? (And the patch name is HIVE-1758 instead of HIVE-1785.) Please resubmit with only the changes relevant to HIVE-1785.
Also, I don't see where you addressed two of my previous comments:
* rewrite existing hook implementations in Hive codebase to implement new interface
* add Javadoc for Hook, HookContext, and ExecuteWithHookContext
One new one while we're at it:
* The old hook interface provided SessionState, Set<ReadEntity>, Set<WriteEntity>, LineageInfo and UserGroupInformation. HookContext should provide getters for these as well. I realize that some of these are obtainable via QueryPlan and HiveConf, but we should not force users to become dependent on internals like QueryPlan unless they want to be; also, providing the equivalent getters makes upgrade to the new interface easier. You can make the HookContext getters call through to QueryPlan getters etc.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive_1758_5.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933044#action_12933044 ]
Namit Jain commented on HIVE-1785:
----------------------------------
Can you regenerate the patch ?
I have already committed HIVE-1642
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1785:
-----------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933062#action_12933062 ]
Namit Jain commented on HIVE-1785:
----------------------------------
+1
running tests
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in
1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HIVE-1785:
-----------------------------
Attachment: hive-1785_6.patch
Thanks for the careful review and sorry to submit the wrong patch before.
This patch makes the all changes according to the discussion before and clears irrelevant code.
Please review.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive-1785_6.patch, hive_1758_5.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935023#action_12935023 ]
John Sichi commented on HIVE-1785:
----------------------------------
+1 on latest. Will commit when tests pass.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive-1785_6.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932175#action_12932175 ]
John Sichi commented on HIVE-1785:
----------------------------------
As mentioned above, we don't want to make a breaking change to the hook interface. To do this, create a new interface PreExecuteWithContext and add the new method there. Then, when invoking hooks, use instanceof to detect whether the class implements PreExecuteWithContext. If not, invoke using the old interface+method.
Repeat for PostExecute.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive_1785_1.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "Liyin Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935186#action_12935186 ]
Liyin Tang commented on HIVE-1785:
----------------------------------
Thanks John's review and I have created a sub task (Hive-1810) to change the xml description.
Please take a look.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.7.0
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Fix For: 0.7.0
>
> Attachments: hive-1785_3.patch, hive-1785_4.patch, hive-1785_6.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take
in 1 parameter: HookContext
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933081#action_12933081 ]
John Sichi commented on HIVE-1785:
----------------------------------
The way the new interface is defined, people writing the hooks will need to keep implementing the old Pre/PostExec interfaces, even though we will no longer be calling those methods. This will make it harder to remove cruft.
> change Pre/Post Query Hooks to take in 1 parameter: HookContext
> ---------------------------------------------------------------
>
> Key: HIVE-1785
> URL: https://issues.apache.org/jira/browse/HIVE-1785
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Liyin Tang
> Attachments: hive-1785_3.patch, hive_1785_1.patch, hive_1785_2.patch
>
>
> This way, it would be possible to add new parameters to the hooks without changing the existing hooks.
> This will be a incompatible change, and all the hooks need to change to the new API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.