You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Mohit Sabharwal <mo...@cloudera.com> on 2015/12/03 09:12:31 UTC
Review Request 40898: HIVE-12431: Support timeout for global compile
lock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40898/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-12431
https://issues.apache.org/jira/browse/HIVE-12431
Repository: hive-git
Description
-------
HIVE-12431: Support timeout for global compile lock
When global (HS2-wide) compile lock is configured, a long-compiling request
will block remaining sessions indefinitely.
This patch allows the user to configure the maximum time a request will wait
to acquire the compile lock. Note that this configuration does not apply when
session scoped compile locking is configured.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f9607129eb1f5f43e8a728cf7d2a56c1ed5af49
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 62b608cbf53c371d1743df40988daf85f76a0867
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 8a47605630066e39272f506c6e309b108b8455dd
service/src/test/org/apache/hive/service/cli/CLIServiceTest.java d90002bd16e46b5ce970d4c6c544a9c7605328d1
Diff: https://reviews.apache.org/r/40898/diff/
Testing
-------
TestEmbeddedThriftBinaryCLIService#testGlobalCompileLockTimeout
Thanks,
Mohit Sabharwal
Re: Review Request 40898: HIVE-12431: Support timeout for global
compile lock
Posted by Mohit Sabharwal <mo...@cloudera.com>.
> On Dec. 3, 2015, 5:40 p.m., Lenni Kuff wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1849
> > <https://reviews.apache.org/r/40898/diff/2/?file=1152868#file1152868line1849>
> >
> > Curious - does it make sense to only apply this to the "global" compile lock? Wouldn't this also be applicable for the session-level compile lock?
Yeah, makes sense. Changed it to apply to either case.
> On Dec. 3, 2015, 5:40 p.m., Lenni Kuff wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/Driver.java, line 139
> > <https://reviews.apache.org/r/40898/diff/2/?file=1152869#file1152869line139>
> >
> > Is there a better way we can add in these test hook (create a mock driver or something)?
Mocking the driver to test concurrent requests is a bit tricky. Cleaned this up by adding a compile time hook instead as suggesed by Sergey.
> On Dec. 3, 2015, 5:40 p.m., Lenni Kuff wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/Driver.java, line 1265
> > <https://reviews.apache.org/r/40898/diff/2/?file=1152869#file1152869line1265>
> >
> > Add a comment to this function to describe what it does and info about the return value.
> >
> > Maybe rename to "tryAcquireCompileLock"?
Fixed.
> On Dec. 3, 2015, 5:40 p.m., Lenni Kuff wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/Driver.java, line 1277
> > <https://reviews.apache.org/r/40898/diff/2/?file=1152869#file1152869line1277>
> >
> > Should this be INFO level? It might be useful to log the query along with this message for debug purposes.
Done.
> On Dec. 3, 2015, 5:40 p.m., Lenni Kuff wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/Driver.java, line 1280
> > <https://reviews.apache.org/r/40898/diff/2/?file=1152869#file1152869line1280>
> >
> > Can we include the query text here?
Done.
- Mohit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40898/#review108847
-----------------------------------------------------------
On Dec. 3, 2015, 8:14 a.m., Mohit Sabharwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40898/
> -----------------------------------------------------------
>
> (Updated Dec. 3, 2015, 8:14 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-12431
> https://issues.apache.org/jira/browse/HIVE-12431
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-12431: Support timeout for global compile lock
>
> When global (HS2-wide) compile lock is configured, a long-compiling request
> will block remaining sessions indefinitely.
>
> This patch allows the user to configure the maximum time a request will wait
> to acquire the compile lock. Note that this configuration does not apply when
> session scoped compile locking is configured.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f9607129eb1f5f43e8a728cf7d2a56c1ed5af49
> ql/src/java/org/apache/hadoop/hive/ql/Driver.java 62b608cbf53c371d1743df40988daf85f76a0867
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 8a47605630066e39272f506c6e309b108b8455dd
> service/src/test/org/apache/hive/service/cli/CLIServiceTest.java d90002bd16e46b5ce970d4c6c544a9c7605328d1
>
> Diff: https://reviews.apache.org/r/40898/diff/
>
>
> Testing
> -------
>
> TestEmbeddedThriftBinaryCLIService#testGlobalCompileLockTimeout
>
>
> Thanks,
>
> Mohit Sabharwal
>
>
Re: Review Request 40898: HIVE-12431: Support timeout for global
compile lock
Posted by Lenni Kuff <ls...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40898/#review108847
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (line 1849)
<https://reviews.apache.org/r/40898/#comment168311>
Curious - does it make sense to only apply this to the "global" compile lock? Wouldn't this also be applicable for the session-level compile lock?
ql/src/java/org/apache/hadoop/hive/ql/Driver.java (line 139)
<https://reviews.apache.org/r/40898/#comment168313>
Is there a better way we can add in these test hook (create a mock driver or something)?
ql/src/java/org/apache/hadoop/hive/ql/Driver.java (line 1265)
<https://reviews.apache.org/r/40898/#comment168314>
Add a comment to this function to describe what it does and info about the return value.
Maybe rename to "tryAcquireCompileLock"?
ql/src/java/org/apache/hadoop/hive/ql/Driver.java (line 1277)
<https://reviews.apache.org/r/40898/#comment168316>
Should this be INFO level? It might be useful to log the query along with this message for debug purposes.
ql/src/java/org/apache/hadoop/hive/ql/Driver.java (line 1280)
<https://reviews.apache.org/r/40898/#comment168318>
Can we include the query text here?
- Lenni Kuff
On Dec. 3, 2015, 8:14 a.m., Mohit Sabharwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40898/
> -----------------------------------------------------------
>
> (Updated Dec. 3, 2015, 8:14 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-12431
> https://issues.apache.org/jira/browse/HIVE-12431
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-12431: Support timeout for global compile lock
>
> When global (HS2-wide) compile lock is configured, a long-compiling request
> will block remaining sessions indefinitely.
>
> This patch allows the user to configure the maximum time a request will wait
> to acquire the compile lock. Note that this configuration does not apply when
> session scoped compile locking is configured.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f9607129eb1f5f43e8a728cf7d2a56c1ed5af49
> ql/src/java/org/apache/hadoop/hive/ql/Driver.java 62b608cbf53c371d1743df40988daf85f76a0867
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 8a47605630066e39272f506c6e309b108b8455dd
> service/src/test/org/apache/hive/service/cli/CLIServiceTest.java d90002bd16e46b5ce970d4c6c544a9c7605328d1
>
> Diff: https://reviews.apache.org/r/40898/diff/
>
>
> Testing
> -------
>
> TestEmbeddedThriftBinaryCLIService#testGlobalCompileLockTimeout
>
>
> Thanks,
>
> Mohit Sabharwal
>
>
Re: Review Request 40898: HIVE-12431: Support timeout for compile lock
Posted by Mohit Sabharwal <mo...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40898/
-----------------------------------------------------------
(Updated Dec. 3, 2015, 11:04 p.m.)
Review request for hive.
Changes
-------
Incorp fb.
Summary (updated)
-----------------
HIVE-12431: Support timeout for compile lock
Bugs: HIVE-12431
https://issues.apache.org/jira/browse/HIVE-12431
Repository: hive-git
Description (updated)
-------
HIVE-12431: Support timeout for compile lock
When compile lock is configured, a long-compiling request will block
remaining sessions or concurrent requests for a given session indefinitely.
This patch allows the user to configure the maximum time a request will wait
to acquire the compile lock.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f9607129eb1f5f43e8a728cf7d2a56c1ed5af49
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 62b608cbf53c371d1743df40988daf85f76a0867
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 8a47605630066e39272f506c6e309b108b8455dd
service/src/test/org/apache/hive/service/cli/CLIServiceTest.java d90002bd16e46b5ce970d4c6c544a9c7605328d1
Diff: https://reviews.apache.org/r/40898/diff/
Testing
-------
TestEmbeddedThriftBinaryCLIService#testGlobalCompileLockTimeout
Thanks,
Mohit Sabharwal
Re: Review Request 40898: HIVE-12431: Support timeout for global
compile lock
Posted by Mohit Sabharwal <mo...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40898/
-----------------------------------------------------------
(Updated Dec. 3, 2015, 8:14 a.m.)
Review request for hive.
Bugs: HIVE-12431
https://issues.apache.org/jira/browse/HIVE-12431
Repository: hive-git
Description
-------
HIVE-12431: Support timeout for global compile lock
When global (HS2-wide) compile lock is configured, a long-compiling request
will block remaining sessions indefinitely.
This patch allows the user to configure the maximum time a request will wait
to acquire the compile lock. Note that this configuration does not apply when
session scoped compile locking is configured.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f9607129eb1f5f43e8a728cf7d2a56c1ed5af49
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 62b608cbf53c371d1743df40988daf85f76a0867
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 8a47605630066e39272f506c6e309b108b8455dd
service/src/test/org/apache/hive/service/cli/CLIServiceTest.java d90002bd16e46b5ce970d4c6c544a9c7605328d1
Diff: https://reviews.apache.org/r/40898/diff/
Testing
-------
TestEmbeddedThriftBinaryCLIService#testGlobalCompileLockTimeout
Thanks,
Mohit Sabharwal