You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by dj...@fb.com on 2011/05/10 23:25:04 UTC
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/
-----------------------------------------------------------
(Updated 2011-05-10 21:25:03.998477)
Review request for hive.
Changes
-------
- First ctrl+c will print a message and try to stop the current command. Second ctrl+c will kill the jvm
- Added code that stops some of the JDO processing
- Now the code can also stop generating input summary.
- Fixed loop in Utilities.getInputSummary that hangs under some conditions
Summary
-------
Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
This addresses bug HIVE-243.
https://issues.apache.org/jira/browse/HIVE-243
Diffs (updated)
-----
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
Diff: https://reviews.apache.org/r/626/diff
Testing
-------
Manual test Ctrl+C on bunch of command (show tables, select, etc)
Thanks,
georgedj
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by Ning Zhang <nz...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/#review673
-----------------------------------------------------------
Ship it!
Looks good to me. George, can you rebase to the latest trunk and upload it to the JIRA? I'll start testing.
- Ning
On 2011-05-16 17:25:41, georgedj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/626/
> -----------------------------------------------------------
>
> (Updated 2011-05-16 17:25:41)
>
>
> Review request for hive.
>
>
> Summary
> -------
>
> Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
>
>
> This addresses bug HIVE-243.
> https://issues.apache.org/jira/browse/HIVE-243
>
>
> Diffs
> -----
>
> trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
>
> Diff: https://reviews.apache.org/r/626/diff
>
>
> Testing
> -------
>
> Manual test Ctrl+C on bunch of command (show tables, select, etc)
> Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
> example queries:
> explain select count(*) from xyz where ds = '2011-02-01';
> select count(*) from xyz where ds = '2011-02-01';
>
>
> Thanks,
>
> georgedj
>
>
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/
-----------------------------------------------------------
(Updated 2011-05-17 18:56:20.310048)
Review request for hive.
Changes
-------
updating diff against the latest trunk
Summary
-------
Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
This addresses bug HIVE-243.
https://issues.apache.org/jira/browse/HIVE-243
Diffs (updated)
-----
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1104456
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1104456
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1104456
Diff: https://reviews.apache.org/r/626/diff
Testing
-------
Manual test Ctrl+C on bunch of command (show tables, select, etc)
Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
example queries:
explain select count(*) from xyz where ds = '2011-02-01';
select count(*) from xyz where ds = '2011-02-01';
Thanks,
georgedj
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/
-----------------------------------------------------------
(Updated 2011-05-16 17:25:41.603093)
Review request for hive.
Changes
-------
- Fixed function name typo.
- Updated inaccurate comment.
- Reverted all changes in ObjectStore.
Summary
-------
Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
This addresses bug HIVE-243.
https://issues.apache.org/jira/browse/HIVE-243
Diffs (updated)
-----
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
Diff: https://reviews.apache.org/r/626/diff
Testing
-------
Manual test Ctrl+C on bunch of command (show tables, select, etc)
Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
example queries:
explain select count(*) from xyz where ds = '2011-02-01';
select count(*) from xyz where ds = '2011-02-01';
Thanks,
georgedj
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
> On 2011-05-14 07:39:31, Ning Zhang wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java, line 1194
> > <https://reviews.apache.org/r/626/diff/3/?file=18468#file18468line1194>
> >
> > We shouldn't check interrupt here since ObjectStore should be run at the metastore server side (if Hive is running non-local mode for metastore). So ^C on CLI won't be detected here. Instead we should check it in HiveMetaStoreClient.java.
> >
> > Since interrupting JDO is quite complicated, I suggest removing the interrupt handling for JDO completely in this JIRA and file a separate JIRA for that.
ok, reverting ObjectStore changes.
> On 2011-05-14 07:39:31, Ning Zhang wrote:
> > trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java, line 326
> > <https://reviews.apache.org/r/626/diff/2-3/?file=17625#file17625line326>
> >
> > Is this comment correct?
Nope, comment fixed.
- georgedj
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/#review670
-----------------------------------------------------------
On 2011-05-10 21:55:56, georgedj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/626/
> -----------------------------------------------------------
>
> (Updated 2011-05-10 21:55:56)
>
>
> Review request for hive.
>
>
> Summary
> -------
>
> Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
>
>
> This addresses bug HIVE-243.
> https://issues.apache.org/jira/browse/HIVE-243
>
>
> Diffs
> -----
>
> trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
>
> Diff: https://reviews.apache.org/r/626/diff
>
>
> Testing
> -------
>
> Manual test Ctrl+C on bunch of command (show tables, select, etc)
> Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
> example queries:
> explain select count(*) from xyz where ds = '2011-02-01';
> select count(*) from xyz where ds = '2011-02-01';
>
>
> Thanks,
>
> georgedj
>
>
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by Ning Zhang <nz...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/#review670
-----------------------------------------------------------
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
<https://reviews.apache.org/r/626/#comment1337>
Is this comment correct?
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java
<https://reviews.apache.org/r/626/#comment1336>
typo: interrup -> interrupt.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/626/#comment1335>
We shouldn't check interrupt here since ObjectStore should be run at the metastore server side (if Hive is running non-local mode for metastore). So ^C on CLI won't be detected here. Instead we should check it in HiveMetaStoreClient.java.
Since interrupting JDO is quite complicated, I suggest removing the interrupt handling for JDO completely in this JIRA and file a separate JIRA for that.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
<https://reviews.apache.org/r/626/#comment1334>
typo: interrup -> interrupt
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
<https://reviews.apache.org/r/626/#comment1338>
why these are removed?
- Ning
On 2011-05-10 21:55:56, georgedj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/626/
> -----------------------------------------------------------
>
> (Updated 2011-05-10 21:55:56)
>
>
> Review request for hive.
>
>
> Summary
> -------
>
> Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
>
>
> This addresses bug HIVE-243.
> https://issues.apache.org/jira/browse/HIVE-243
>
>
> Diffs
> -----
>
> trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
>
> Diff: https://reviews.apache.org/r/626/diff
>
>
> Testing
> -------
>
> Manual test Ctrl+C on bunch of command (show tables, select, etc)
> Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
> example queries:
> explain select count(*) from xyz where ds = '2011-02-01';
> select count(*) from xyz where ds = '2011-02-01';
>
>
> Thanks,
>
> georgedj
>
>
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/#review671
-----------------------------------------------------------
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
<https://reviews.apache.org/r/626/#comment1339>
The loop wrongly assumes that calling Thread.currentThread().interrupt() will throw ThreadInterrupted while it only sets interrupted flag on the thread. The result is infinite loop with executionDone never set to true. The loop is actually not needed as executor.shutdown() does the same thing - waits for all queued jobs to complete.
- georgedj
On 2011-05-10 21:55:56, georgedj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/626/
> -----------------------------------------------------------
>
> (Updated 2011-05-10 21:55:56)
>
>
> Review request for hive.
>
>
> Summary
> -------
>
> Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
>
>
> This addresses bug HIVE-243.
> https://issues.apache.org/jira/browse/HIVE-243
>
>
> Diffs
> -----
>
> trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
> trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
>
> Diff: https://reviews.apache.org/r/626/diff
>
>
> Testing
> -------
>
> Manual test Ctrl+C on bunch of command (show tables, select, etc)
> Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
> example queries:
> explain select count(*) from xyz where ds = '2011-02-01';
> select count(*) from xyz where ds = '2011-02-01';
>
>
> Thanks,
>
> georgedj
>
>
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/
-----------------------------------------------------------
(Updated 2011-05-10 21:55:56.707055)
Review request for hive.
Changes
-------
Fixed some white space formatting
Summary
-------
Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
This addresses bug HIVE-243.
https://issues.apache.org/jira/browse/HIVE-243
Diffs (updated)
-----
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
Diff: https://reviews.apache.org/r/626/diff
Testing
-------
Manual test Ctrl+C on bunch of command (show tables, select, etc)
Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
example queries:
explain select count(*) from xyz where ds = '2011-02-01';
select count(*) from xyz where ds = '2011-02-01';
Thanks,
georgedj
Re: Review Request: HIVE-243 - ^C breaks out of running query,
but not whole CLI.
Posted by dj...@fb.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/626/
-----------------------------------------------------------
(Updated 2011-05-10 21:33:17.693931)
Review request for hive.
Summary
-------
Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.
This addresses bug HIVE-243.
https://issues.apache.org/jira/browse/HIVE-243
Diffs
-----
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312
Diff: https://reviews.apache.org/r/626/diff
Testing (updated)
-------
Manual test Ctrl+C on bunch of command (show tables, select, etc)
Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
example queries:
explain select count(*) from xyz where ds = '2011-02-01';
select count(*) from xyz where ds = '2011-02-01';
Thanks,
georgedj