[jira] [Resolved] (IMPALA-13631) alterTableOrViewRename shouldn't hold catalogVersionLock during external RPCs

["Michael Smith (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Smith resolved IMPALA-13631.
------------------------------------
    Fix Version/s: Impala 5.0.0
       Resolution: Fixed

> alterTableOrViewRename shouldn't hold catalogVersionLock during external RPCs
> -----------------------------------------------------------------------------
>
>                 Key: IMPALA-13631
>                 URL: https://issues.apache.org/jira/browse/IMPALA-13631
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Critical
>             Fix For: Impala 5.0.0
>
>
> CatalogOpExecutor.alterTableOrViewRename() requires holding the catalogVersion writeLock, as the comment mentioned:
> {code:java}
>         // RENAME is implemented as an ADD + DROP, so we need to execute it as we hold
>         // the catalog lock.
>         try {
>           alterTableOrViewRename(tbl,
>               TableName.fromThrift(params.getRename_params().getNew_table_name()),
>               modification, wantMinimalResult, response, catalogTimeline);
>           modification.validateInProgressModificationComplete();
>           return;
>         } finally {
>           // release the version taken in the tryLock call above
>           catalog_.getLock().writeLock().unlock();
>         } {code}
> https://github.com/apache/impala/blob/0bbd2b684ddc7dcf8b6c16f1f7c6fab15291f782/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1221-L1232
> However, alterTableOrViewRename() triggers external RPCs, e.g. HMS alter_table RPC, which could hang due to external issues. Holding the catalogVersion writeLock blocks all other catalog operations, including all the read requests like getPartialCatalogObject or collecting catalog updates. This will impact the whole Impala cluster. Lots of queries will be blocked in the CREATED state.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)