You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "BELUGA BEHR (JIRA)" <ji...@apache.org> on 2019/02/28 16:19:00 UTC
[jira] [Created] (HIVE-21354) Lock The Entire Table If Majority Of
Partitions Are Locked
BELUGA BEHR created HIVE-21354:
----------------------------------
Summary: Lock The Entire Table If Majority Of Partitions Are Locked
Key: HIVE-21354
URL: https://issues.apache.org/jira/browse/HIVE-21354
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Affects Versions: 4.0.0, 3.2.0
Reporter: BELUGA BEHR
One of the bottlenecks of any Hive query is the ZooKeeper locking mechanism. When a Hive query interacts with a table which has a lot of partitions, this may put a lot of stress on the ZK system.
Please add a heuristic that works like this:
# Count the number of partitions that a query is required to lock
# Obtain the total number of partitions in the table
# If the number of partitions accessed by the query is greater than or equal to half the total number of partitions, simply create one ZNode lock at the table level.
This would improve performance of many queries, but in particular, a {{select count(1) from table}} ... or ... {{select * from table limit 5}} where the table has many partitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)