[jira] [Created] (IMPALA-10082) Concurrent invalidate metadata and create/drop table cause discrepancy in metadata

["Robbie Zhang (Jira)" <ji...@apache.org>]

Robbie Zhang created IMPALA-10082:
-------------------------------------

             Summary: Concurrent invalidate metadata and create/drop table cause discrepancy in metadata
                 Key: IMPALA-10082
                 URL: https://issues.apache.org/jira/browse/IMPALA-10082
             Project: IMPALA
          Issue Type: Bug
    Affects Versions: Impala 4.0
            Reporter: Robbie Zhang


The symptom is similar to IMPALA-7093 but is a different issue. Here is how I reproduce it:

1) Ran the first script to keep running create/insert/drop queries
{code:java}
#!/bin/bash

while [ 1 ]
do
  shell/impala-shell -q "create table if not exists test(i int); insert into test(i) values(1); drop table test;" 2>&1| tee test.output
  n=`egrep "Exception" test.output | wc -l`
  if [ $n -lt 0 ]; then
    rm -f /tmp/testing
    exit
  fi
done
{code}
2) Ran the second script to keep running global invalidate metadata
{code:java}
#!/bin/bash

while [ 1 ]
do
  shell/impala-shell -q "invalidate metadata"
done
{code}
Sometime later, the first scrip ended with "Table default.test does not exist":
{code:java}
Starting Impala Shell with no authentication using Python 2.7.12
Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
Opened TCP connection to localhost:21000
Connected to localhost:21000
Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build f95f7940e4a290d75ee85fd78e85bc26795f0f9f)
Query: create table if not exists test(i int)
Fetched 1 row(s) in 0.01s
Query: insert into test(i) values(1)
Query submitted at: 2020-08-13 22:57:51 (Coordinator: http://impala34:25000)
ERROR: AnalysisException: Table does not exist: default.test


Could not execute command: insert into test(i) values(1){code}
Even after I change to local catalog mode, this issue still exists:
{code:java}
Starting Impala Shell with no authentication using Python 2.7.12
Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
Opened TCP connection to localhost:21000
Connected to localhost:21000
Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build f95f7940e4a290d75ee85fd78e85bc26795f0f9f)
Query: create table if not exists test(i int)
Fetched 1 row(s) in 0.07s
Query: insert into test(i) values(1)
Query submitted at: 2020-08-13 22:10:16 (Coordinator: http://impala34:25000)
ERROR: AnalysisException: org.apache.impala.catalog.TableLoadingException: Could not load table default.test from catalog
CAUSED BY: TableLoadingException: Could not load table default.test from catalog
CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[TableLoadingException: Table default.test no longer exists in the Hive MetaStore. Run 'invalidate metadata default.test' to update the Impala catalog.]), lookup_status:OK)


Could not execute command: insert into test(i) values(1)
{code}
And in local catalog mode, the newly created table was lost but it's still visible in the coordinator. After running 'invalidate metadata default.test', the table disappeared at all.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org