You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by ch...@huawei.com on 2011/08/22 20:21:29 UTC
Review Request: removed system.exit() from Driver.taskCleanup() and added
code to kill the remaining tasks
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1609/
-----------------------------------------------------------
Review request for hive, John Sichi and Ning Zhang.
Summary
-------
while processing two parallel tasks if one of the task fails the Driver.taskCleanup() will call system.exit() this will shutdown the jvm so it is replaced with the logic to stop the remaining tasks.. Here need to cleanup 2 kinds of tasks one is non-mr tasks and mr tasks. For stopping non-mr tasks used the thread.interrupt() because every non-mr task will be executed as a thread and for mr tasks maintained a variable called jobKillUri's this variable will track the spawned job kill uri, in taskCleanup() if it is a mr task using this variable kill the job.
This addresses bug HIVE-2017.
https://issues.apache.org/jira/browse/HIVE-2017
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1160102
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1160102
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1160102
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1160102
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1160102
trunk/ql/src/test/queries/clientnegative/alter_exit.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/alter_exit.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1609/diff
Testing
-------
Added test case for this scenario. Ran existing test cases
Thanks,
chinna
Re: Review Request: removed system.exit() from Driver.taskCleanup() and
added code to kill the remaining tasks
Posted by ch...@huawei.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1609/
-----------------------------------------------------------
(Updated 2011-11-03 16:01:16.713162)
Review request for hive, John Sichi and Ning Zhang.
Changes
-------
Patch becomes stale so rebased
Summary
-------
while processing two parallel tasks if one of the task fails the Driver.taskCleanup() will call system.exit() this will shutdown the jvm so it is replaced with the logic to stop the remaining tasks.. Here need to cleanup 2 kinds of tasks one is non-mr tasks and mr tasks. For stopping non-mr tasks used the thread.interrupt() because every non-mr task will be executed as a thread and for mr tasks maintained a variable called jobKillUri's this variable will track the spawned job kill uri, in taskCleanup() if it is a mr task using this variable kill the job.
This addresses bug HIVE-2017.
https://issues.apache.org/jira/browse/HIVE-2017
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1197183
trunk/ql/src/test/queries/clientnegative/alter_exit.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/alter_exit.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1609/diff
Testing
-------
Added test case for this scenario. Ran existing test cases
Thanks,
chinna
Re: Review Request: removed system.exit() from Driver.taskCleanup() and
added code to kill the remaining tasks
Posted by ch...@huawei.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1609/
-----------------------------------------------------------
(Updated 2011-09-16 17:33:41.236186)
Review request for hive, John Sichi and Ning Zhang.
Changes
-------
Rebased the patch on the latest trunk
Summary
-------
while processing two parallel tasks if one of the task fails the Driver.taskCleanup() will call system.exit() this will shutdown the jvm so it is replaced with the logic to stop the remaining tasks.. Here need to cleanup 2 kinds of tasks one is non-mr tasks and mr tasks. For stopping non-mr tasks used the thread.interrupt() because every non-mr task will be executed as a thread and for mr tasks maintained a variable called jobKillUri's this variable will track the spawned job kill uri, in taskCleanup() if it is a mr task using this variable kill the job.
This addresses bug HIVE-2017.
https://issues.apache.org/jira/browse/HIVE-2017
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1170977
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1170977
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1170977
trunk/ql/src/test/queries/clientnegative/alter_exit.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/alter_exit.q.out PRE-CREATION
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1170977
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1170977
Diff: https://reviews.apache.org/r/1609/diff
Testing
-------
Added test case for this scenario. Ran existing test cases
Thanks,
chinna