You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Yongzhi Chen (JIRA)" <ji...@apache.org> on 2017/01/11 14:24:58 UTC
[jira] [Commented] (HIVE-15572) Improve the response time for query
canceling when it happens during acquiring locks
[ https://issues.apache.org/jira/browse/HIVE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818466#comment-15818466 ]
Yongzhi Chen commented on HIVE-15572:
-------------------------------------
1) Add data structure to pass driverstate around.
2) Improve ZooKeeperHiveLockManager response time for canceling by checking driver state before each acquiring lock call to zookeeper.
> Improve the response time for query canceling when it happens during acquiring locks
> ------------------------------------------------------------------------------------
>
> Key: HIVE-15572
> URL: https://issues.apache.org/jira/browse/HIVE-15572
> Project: Hive
> Issue Type: Improvement
> Reporter: Yongzhi Chen
> Assignee: Yongzhi Chen
>
> When query canceling command sent during Hive Acquire locks (from zookeeper), hive will finish acquiring all the locks and release them. As it is shown in the following log:
> It took 165 s to finish acquire the lock,then spend 81s to release them.
> We can improve the performance by not acquiring any more locks and releasing held locks when the query canceling command is received.
> {noformat}
> Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:50:35,413 INFO org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-224]: <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,671 INFO org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: </PERFLOG method=acquireReadWriteLocks start=1483469295080 end=1483469460671 duration=165591 from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,672 INFO org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from manyparttbl has been cancelled
> 2017-01-03 10:51:00,673 INFO org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:40,755 INFO org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks start=1483469419487 end=1483469500755 duration=81268 from=org.apache.hadoop.hive.ql.Driver>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)