You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/04/07 02:20:00 UTC
[jira] [Commented] (IMPALA-11153) Expose LOCK_RETRIES/LOCK_RETRY_WAIT_SECONDS settings for user
[ https://issues.apache.org/jira/browse/IMPALA-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518544#comment-17518544 ]
ASF subversion and git services commented on IMPALA-11153:
----------------------------------------------------------
Commit a0922079f5b0ba201025b0b657371f26bc2a7d16 in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a0922079f ]
IMPALA-11153: Make lock wait time configurable for the users
Currently Impala is using private static final values for lock
retry/wait:
// Number of retries to acquire an HMS ACID lock.
private static final int LOCK_RETRIES = 10;
// Time interval between retries of acquiring an HMS ACID lock
private static final int LOCK_RETRY_WAIT_SECONDS = 3;
This patch changes the logic for waiting on locks. It does an
exponential backoff starting from 50 ms up to 30 seconds, and the
users can configure a maximum total wait time for the locks. This
total maximum wait time is 5 minutes (the earlier 30 seconds
was too short in real use cases).
Testing:
* added e2e tests
Change-Id: I055b76138dd30b2c40eedb48905cb3bade1438fc
Reviewed-on: http://gerrit.cloudera.org:8080/18289
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Expose LOCK_RETRIES/LOCK_RETRY_WAIT_SECONDS settings for user
> -------------------------------------------------------------
>
> Key: IMPALA-11153
> URL: https://issues.apache.org/jira/browse/IMPALA-11153
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: ramp-up
>
> Impala using private static final value for lock retry/wait.
> https://github.com/apache/impala/blob/873fe2e5241c5714dfd94a186d524edc1cbad0ad/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java#L142
> {noformat}
> // Number of retries to acquire an HMS ACID lock.
> private static final int LOCK_RETRIES = 10;
> // Time interval between retries of acquiring an HMS ACID lock
> private static final int LOCK_RETRY_WAIT_SECONDS = 3;
> {noformat}
> In some cases it would make sense to make these values tunable.
> We invoke acquireLock() in these places:
> https://github.infra.cloudera.com/CDH/Impala/blob/3ceabc01822f53bb6f174dc29aac56ac3993d389/fe/src/main/java/org/apache/impala/service/Frontend.java#L2169
> https://github.infra.cloudera.com/CDH/Impala/blob/7ae6fff71c860f7623fc0bfd3092073128c4959c/fe/src/main/java/org/apache/impala/catalog/Catalog.java#L745
> invoked by
> https://github.infra.cloudera.com/CDH/Impala/blob/ac62b98ab1d3475d8ece865c67fe90d463c340b5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1936
> In Frontend.java it's easy to access the query options. To have the tunables at the CatalogD -side we probably need to extend TDropTableOrViewParams or TDdlExecRequest.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org