[jira] [Created] (IMPALA-9101) Unneccessary REFRESH due to wrong self-event detection

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-9101:
--------------------------------------

             Summary: Unneccessary REFRESH due to wrong self-event detection
                 Key: IMPALA-9101
                 URL: https://issues.apache.org/jira/browse/IMPALA-9101
             Project: IMPALA
          Issue Type: Bug
            Reporter: Quanlong Huang


In {{CatalogOpExecutor.alterTable()}}, we call {{addVersionsForInflightEvents()}} whenever the AlterTable operation changes anything or not. If nothing changes, no HMS RPCs are sent. The event processor ends up waiting on a non-existed self-event. Then all self-events are treated as outside events and unneccessary REFRESH/INVALIDATE on this table will be performed.

Codes:
{code:java}
  private void alterTable(TAlterTableParams params, TDdlExecResponse response)
      throws ImpalaException {
    ....
    tryLock(tbl);
    // Get a new catalog version to assign to the table being altered.
    long newCatalogVersion = catalog_.incrementAndGetCatalogVersion();
    addCatalogServiceIdentifiers(tbl, catalog_.getCatalogServiceId(), newCatalogVersion);
    ....
      // now that HMS alter operation has succeeded, add this version to list of inflight
      // events in catalog table if event processing is enabled
      catalog_.addVersionsForInflightEvents(tbl, newCatalogVersion);    <---- We should check before calling this.
  }
{code}
Reproduce:
{code:sql}
create table testtbl (col int) partitioned by (p1 int, p2 int);
alter table testtbl add partition (p1=2,p2=6);
alter table testtbl add if not exists partition (p1=2,p2=6);
-- After this point, can't detect self-events on this table
alter table testtbl add partition (p1=2,p2=7);
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)