[Impala-ASF-CR] IMPALA-6671: Skip locked tables from topic updates

["Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>]

Vihang Karajgaonkar has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/16549 )

Change subject: IMPALA-6671: Skip locked tables from topic updates
......................................................................

IMPALA-6671: Skip locked tables from topic updates

This change adds a mechanism for topic-update thread
to skip a table which is locked for more than a
configurable interval from the topic updates. This is
especially useful in scenarios where long running operations on a
locked table (refresh, recover partitions, compute stats) block the
topic update thread. This causes unrelated queries which are
waiting on metadata via topic updates (catalog-v1 mode)
to unnecessarily block.

The ideal solution of this problem would be to make HdfsTable
immutable so that there is no need for table lock. But that
is large change and not easily portable to older releases
of Impala. It would be taken up as a separate patch.

This change introduces 2 new configurations for catalogd:

1. topic_update_tbl_max_wait_time_ms: This defines the
maximum time in msecs the topic update thread waits on a locked table
before skipping the table from that iteration of topic updates.
The default value is 500.
2. catalog_max_lock_skipped_topic_updates: This defines
the maximum number of distinct lock operations which are skipped
by topic update thread due to lock contention. Once this limit
is reached, topic update thread will block until it acquires
the table lock and adds it to the updates.

Testing:
1. Added a test case which introduces a simulated delay
in a few potentially long running statements. This causes the table
to be locked for a long time. The topic update thread skips
that table from updates and unrelated queries are unblocked
since they receive the required metadata from updates.
2. Added a test where multiple threads run blocking statements
in a loop to stress the table lock. It makes sure that topic
update thread is not starved and eventually blocks
on table lock by hitting the limit defined by
catalog_max_lock_skipped_topic_updates.
2. Ran exhaustive tests with default configurations.

Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655
---
M be/src/catalog/catalog-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/analysis/CopyTestCaseStmt.java
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogObjectToFromThriftTest.java
A tests/metadata/test_topic_update_frequency.py
13 files changed, 647 insertions(+), 149 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/16549/10
-- 
To view, visit http://gerrit.cloudera.org:8080/16549
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655
Gerrit-Change-Number: 16549
Gerrit-PatchSet: 10
Gerrit-Owner: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>