You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Chaoyu Tang <ct...@gmail.com> on 2014/07/22 21:21:41 UTC
Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is not
able to acquire locks
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/
-----------------------------------------------------------
Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
Bugs: HIVE-7445
https://issues.apache.org/jira/browse/HIVE-7445
Repository: hive-git
Description
-------
This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
ql/src/test/results/clientnegative/insert_into1.q.out a38b679
ql/src/test/results/clientnegative/insert_into2.q.out f21823a
ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
Diff: https://reviews.apache.org/r/23820/diff/
Testing
-------
1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
---
Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
---
2. Precommit tests
The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
Thanks,
Chaoyu Tang
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Szehon Ho <sz...@cloudera.com>.
> On July 22, 2014, 10:33 p.m., Szehon Ho wrote:
> > Hey Chaoyu thanks, the logic looks good now. Can you also fix all the red-space? It's two lines per indent in hive, thanks.
Just realized I had a typo, its two spaces per indent.
- Szehon
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/#review48441
-----------------------------------------------------------
On July 22, 2014, 7:21 p.m., Chaoyu Tang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23820/
> -----------------------------------------------------------
>
> (Updated July 22, 2014, 7:21 p.m.)
>
>
> Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-7445
> https://issues.apache.org/jira/browse/HIVE-7445
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
> 1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
> 2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
>
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
> ql/src/test/results/clientnegative/insert_into1.q.out a38b679
> ql/src/test/results/clientnegative/insert_into2.q.out f21823a
> ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
> ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
> ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
> ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
> ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
> ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
>
> Diff: https://reviews.apache.org/r/23820/diff/
>
>
> Testing
> -------
>
> 1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
> ---
> Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
> ---
>
> 2. Precommit tests
> The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
>
>
> Thanks,
>
> Chaoyu Tang
>
>
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Chaoyu Tang <ct...@gmail.com>.
> On July 22, 2014, 10:33 p.m., Szehon Ho wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java, line 300
> > <https://reviews.apache.org/r/23820/diff/1/?file=639471#file639471line300>
> >
> > Why was this necessary to change?
Hey, Szehon, Thanks for the review. I think this change is necessary to make the counting logic for lock attempt more accurate and cleaner. The previous counting logic might have a small bug. For example, if we set hive.lock.numretries (The number of times you want to try to get all the locks) to 2, it only allows you to make one locking attempt and then loop out if not successful. Actually the counting logic used in method unlockWithRetry is right and the one used here is same as that. Please let me know if anything I am missing here.
- Chaoyu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/#review48441
-----------------------------------------------------------
On July 23, 2014, 2:27 a.m., Chaoyu Tang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23820/
> -----------------------------------------------------------
>
> (Updated July 23, 2014, 2:27 a.m.)
>
>
> Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-7445
> https://issues.apache.org/jira/browse/HIVE-7445
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
> 1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
> 2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
>
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
> ql/src/test/results/clientnegative/insert_into1.q.out a38b679
> ql/src/test/results/clientnegative/insert_into2.q.out f21823a
> ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
> ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
> ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
> ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
> ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
> ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
>
> Diff: https://reviews.apache.org/r/23820/diff/
>
>
> Testing
> -------
>
> 1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
> ---
> Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
> ---
>
> 2. Precommit tests
> The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
>
>
> Thanks,
>
> Chaoyu Tang
>
>
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Szehon Ho <sz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/#review48441
-----------------------------------------------------------
Hey Chaoyu thanks, the logic looks good now. Can you also fix all the red-space? It's two lines per indent in hive, thanks.
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
<https://reviews.apache.org/r/23820/#comment85045>
Why was this necessary to change?
- Szehon Ho
On July 22, 2014, 7:21 p.m., Chaoyu Tang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23820/
> -----------------------------------------------------------
>
> (Updated July 22, 2014, 7:21 p.m.)
>
>
> Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-7445
> https://issues.apache.org/jira/browse/HIVE-7445
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
> 1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
> 2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
>
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
> ql/src/test/results/clientnegative/insert_into1.q.out a38b679
> ql/src/test/results/clientnegative/insert_into2.q.out f21823a
> ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
> ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
> ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
> ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
> ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
> ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
>
> Diff: https://reviews.apache.org/r/23820/diff/
>
>
> Testing
> -------
>
> 1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
> ---
> Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
> ---
>
> 2. Precommit tests
> The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
>
>
> Thanks,
>
> Chaoyu Tang
>
>
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Lars Francke <as...@lars-francke.de>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/#review48483
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
<https://reviews.apache.org/r/23820/#comment85125>
Missing space after "try"
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
<https://reviews.apache.org/r/23820/#comment85126>
Remove the extra comma at the end
- Lars Francke
On July 23, 2014, 2:27 a.m., Chaoyu Tang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23820/
> -----------------------------------------------------------
>
> (Updated July 23, 2014, 2:27 a.m.)
>
>
> Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-7445
> https://issues.apache.org/jira/browse/HIVE-7445
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
> 1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
> 2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
>
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
> ql/src/test/results/clientnegative/insert_into1.q.out a38b679
> ql/src/test/results/clientnegative/insert_into2.q.out f21823a
> ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
> ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
> ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
> ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
> ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
> ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
>
> Diff: https://reviews.apache.org/r/23820/diff/
>
>
> Testing
> -------
>
> 1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
> ---
> Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
> ---
>
> 2. Precommit tests
> The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
>
>
> Thanks,
>
> Chaoyu Tang
>
>
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Szehon Ho <sz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/#review48601
-----------------------------------------------------------
Ship it!
- Szehon Ho
On July 23, 2014, 12:38 p.m., Chaoyu Tang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23820/
> -----------------------------------------------------------
>
> (Updated July 23, 2014, 12:38 p.m.)
>
>
> Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-7445
> https://issues.apache.org/jira/browse/HIVE-7445
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
> 1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
> 2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
>
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
> ql/src/test/results/clientnegative/insert_into1.q.out a38b679
> ql/src/test/results/clientnegative/insert_into2.q.out f21823a
> ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
> ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
> ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
> ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
> ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
> ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
>
> Diff: https://reviews.apache.org/r/23820/diff/
>
>
> Testing
> -------
>
> 1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
> ---
> Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
> 14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
> ---
>
> 2. Precommit tests
> The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
>
>
> Thanks,
>
> Chaoyu Tang
>
>
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Chaoyu Tang <ct...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/
-----------------------------------------------------------
(Updated July 23, 2014, 12:38 p.m.)
Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
Changes
-------
two more format changes.
Bugs: HIVE-7445
https://issues.apache.org/jira/browse/HIVE-7445
Repository: hive-git
Description
-------
This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
ql/src/test/results/clientnegative/insert_into1.q.out a38b679
ql/src/test/results/clientnegative/insert_into2.q.out f21823a
ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
Diff: https://reviews.apache.org/r/23820/diff/
Testing
-------
1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
---
Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
---
2. Precommit tests
The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
Thanks,
Chaoyu Tang
Re: Review Request 23820: HIVE-7445:Improve LOGS for Hive when a query is
not able to acquire locks
Posted by Chaoyu Tang <ct...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23820/
-----------------------------------------------------------
(Updated July 23, 2014, 2:27 a.m.)
Review request for hive, Brock Noland, Prasad Mujumdar, Szehon Ho, and Xuefu Zhang.
Changes
-------
1. remove all red blocks and make sure the indention is 2 empty space.
Bugs: HIVE-7445
https://issues.apache.org/jira/browse/HIVE-7445
Repository: hive-git
Description
-------
This patch enables ZookeeperHiveLockManager.java, when in debug mode, to log out information about contentious locks if lockmgr fails to acquire a lock for a query. The changes include:
1. Collect and log out contentious lock information in ZookeeperHiveLockManager.java
2. Improve lock retry counting and existing logging, and modified some qtests output files to match these logging
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 1cc3074
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 65b9136
ql/src/test/results/clientnegative/insert_into1.q.out a38b679
ql/src/test/results/clientnegative/insert_into2.q.out f21823a
ql/src/test/results/clientnegative/insert_into3.q.out 5f1581e
ql/src/test/results/clientnegative/insert_into4.q.out 5dcdd50
ql/src/test/results/clientnegative/lockneg1.q.out 6a76cd7
ql/src/test/results/clientnegative/lockneg_try_db_lock_conflict.q.out a9833a8
ql/src/test/results/clientnegative/lockneg_try_drop_locked_db.q.out d67365a
ql/src/test/results/clientnegative/lockneg_try_lock_db_in_use.q.out 89d3265
Diff: https://reviews.apache.org/r/23820/diff/
Testing
-------
1. Manual tests: following debug logging was printed out as expected during a lock acquisition failure.
---
Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 ERROR ZooKeeperHiveLockManager: Unable to acquire IMPLICIT, EXCLUSIVE lock default@sample_07 after 2 attempts.
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Requested lock default@sample_07:: mode:IMPLICIT,EXCLUSIVE; query:insert into table sample_07 select * from sample_08
14/07/22 11:44:40 DEBUG ZooKeeperHiveLockManager: Conflicting lock to default@sample_07:: mode:IMPLICIT;query:select code from sample_07;queryId:root_20140722084242_98f8d9d7-d110-45c0-8c8b-12da2e5172d9;clientIp:10.20.92.233
---
2. Precommit tests
The test failure testCliDriver_ql_rewrite_gbtoidx is a preexisting one and I think it is not related to this change.
Thanks,
Chaoyu Tang