You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (JIRA)" <ji...@apache.org> on 2011/08/04 19:32:27 UTC
[jira] [Created] (HIVE-2346) Add hooks to run when execution fails.
Add hooks to run when execution fails.
--------------------------------------
Key: HIVE-2346
URL: https://issues.apache.org/jira/browse/HIVE-2346
Project: Hive
Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
Currently, when a query fails, the Post Execution Hooks are not run.
Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2346) Add hooks to run when execution fails.
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2346:
--------------------------------
Attachment: HIVE-2346.2.patch.txt
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079534#comment-13079534 ]
jiraposter@reviews.apache.org commented on HIVE-2346:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1295/
-----------------------------------------------------------
Review request for hive and Ning Zhang.
Summary
-------
I added a new type of hook, which will be run when a job fails.
This addresses bug HIVE-2346.
https://issues.apache.org/jira/browse/HIVE-2346
Diffs
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 1153966
Diff: https://reviews.apache.org/r/1295/diff
Testing
-------
I ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
In addition, I created a sample hook, which simply logged that it was run. I verified it was run on a failure, but not when a job succeeded.
Thanks,
Kevin
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang resolved HIVE-2346.
------------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Committed. Thanks Kevin!
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.8.0
>
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt, HIVE-2346.3.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082145#comment-13082145 ]
Ning Zhang commented on HIVE-2346:
----------------------------------
+1. Will commit if tests pass.
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt, HIVE-2346.3.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079533#comment-13079533 ]
Kevin Wilfong commented on HIVE-2346:
-------------------------------------
https://reviews.apache.org/r/1295/
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2346) Add hooks to run when execution fails.
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2346:
--------------------------------
Attachment: HIVE-2346.1.patch.txt
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081998#comment-13081998 ]
jiraposter@reviews.apache.org commented on HIVE-2346:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1295/
-----------------------------------------------------------
(Updated 2011-08-09 23:44:00.972625)
Review request for hive and Ning Zhang.
Changes
-------
Added the hive.exec.failure.hooks property to hive-default.xml
Summary
-------
I added a new type of hook, which will be run when a job fails.
This addresses bug HIVE-2346.
https://issues.apache.org/jira/browse/HIVE-2346
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1155569
trunk/conf/hive-default.xml 1155569
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1155569
trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 1155569
Diff: https://reviews.apache.org/r/1295/diff
Testing
-------
I ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
In addition, I created a sample hook, which simply logged that it was run. I verified it was run on a failure, but not when a job succeeded.
Thanks,
Kevin
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt, HIVE-2346.3.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081944#comment-13081944 ]
jiraposter@reviews.apache.org commented on HIVE-2346:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1295/#review1364
-----------------------------------------------------------
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
<https://reviews.apache.org/r/1295/#comment3082>
As a convention, we should declare the variable as interface (List in this case) rather than implementation (ArrayList).
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
<https://reviews.apache.org/r/1295/#comment3086>
comments are not correct: post execution --> failure
- Ning
On 2011-08-04 19:06:43, Kevin Wilfong wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1295/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-04 19:06:43)
bq.
bq.
bq. Review request for hive and Ning Zhang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I added a new type of hook, which will be run when a job fails.
bq.
bq.
bq. This addresses bug HIVE-2346.
bq. https://issues.apache.org/jira/browse/HIVE-2346
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153966
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153966
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 1153966
bq.
bq. Diff: https://reviews.apache.org/r/1295/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
bq.
bq. In addition, I created a sample hook, which simply logged that it was run. I verified it was run on a failure, but not when a job succeeded.
bq.
bq.
bq. Thanks,
bq.
bq. Kevin
bq.
bq.
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2346) Add hooks to run when execution fails.
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2346:
--------------------------------
Attachment: HIVE-2346.3.patch.txt
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt, HIVE-2346.3.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081993#comment-13081993 ]
jiraposter@reviews.apache.org commented on HIVE-2346:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1295/#review1370
-----------------------------------------------------------
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/1295/#comment3144>
Kevin, can you add this parameter to conf/hive-default.xml? We should add all newly introduced parameters to that file.
Otherwise it looks good to me.
- Ning
On 2011-08-09 22:01:28, Kevin Wilfong wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1295/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-09 22:01:28)
bq.
bq.
bq. Review request for hive and Ning Zhang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I added a new type of hook, which will be run when a job fails.
bq.
bq.
bq. This addresses bug HIVE-2346.
bq. https://issues.apache.org/jira/browse/HIVE-2346
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1155569
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1155569
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 1155569
bq.
bq. Diff: https://reviews.apache.org/r/1295/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
bq.
bq. In addition, I created a sample hook, which simply logged that it was run. I verified it was run on a failure, but not when a job succeeded.
bq.
bq.
bq. Thanks,
bq.
bq. Kevin
bq.
bq.
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083013#comment-13083013 ]
Hudson commented on HIVE-2346:
------------------------------
Integrated in Hive-trunk-h0.21 #887 (See [https://builds.apache.org/job/Hive-trunk-h0.21/887/])
HIVE-2346. Add hooks to run when execution fails. (Kevin Wilfong via Ning Zhang)
nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1156480
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
* /hive/trunk/conf/hive-default.xml
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.8.0
>
> Attachments: HIVE-2346.1.patch.txt, HIVE-2346.2.patch.txt, HIVE-2346.3.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2346) Add hooks to run when execution
fails.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081960#comment-13081960 ]
jiraposter@reviews.apache.org commented on HIVE-2346:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1295/
-----------------------------------------------------------
(Updated 2011-08-09 22:01:28.619424)
Review request for hive and Ning Zhang.
Changes
-------
Made the changes requested by nzhang.
Regarding the change ArrayList --> List, I had copied that code from the analogous methods for Post and Pre Exec hooks. So I made the corresponding changes in those methods too.
Summary
-------
I added a new type of hook, which will be run when a job fails.
This addresses bug HIVE-2346.
https://issues.apache.org/jira/browse/HIVE-2346
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1155569
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1155569
trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 1155569
Diff: https://reviews.apache.org/r/1295/diff
Testing
-------
I ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
In addition, I created a sample hook, which simply logged that it was run. I verified it was run on a failure, but not when a job succeeded.
Thanks,
Kevin
> Add hooks to run when execution fails.
> --------------------------------------
>
> Key: HIVE-2346
> URL: https://issues.apache.org/jira/browse/HIVE-2346
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2346.1.patch.txt
>
>
> Currently, when a query fails, the Post Execution Hooks are not run.
> Adding hooks to be run when a query fails could allow for better logging etc.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira