You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2018/09/05 19:50:00 UTC
[jira] [Commented] (IMPALA-7509) Create table after drop can lead
to table not found exception
[ https://issues.apache.org/jira/browse/IMPALA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604846#comment-16604846 ]
Todd Lipcon commented on IMPALA-7509:
-------------------------------------
The issue here is due to a general state that we can get into in fetch-from-catalogd mode:
- the cache has a table list including some table 't'
- that table has been deleted on the catalogd, but hasn't yet propagated the invalidation to the catalogd
- try to access that table
In this case the table will "exist" (db.getTable(t) will return not null) but then when you attempt to load the table, you'll get a "not found" error when attempting to fetch it. We probably need to catch such an error and ensure that we return null from 'getTable' or throw an InconsistentMetadataException triggering a replan.
> Create table after drop can lead to table not found exception
> -------------------------------------------------------------
>
> Key: IMPALA-7509
> URL: https://issues.apache.org/jira/browse/IMPALA-7509
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Rituparna Agrawal
> Priority: Critical
> Attachments: failure_snippet.txt
>
>
> There are two impalads. One running with old mode and one with fetch from catalogd mode.
> Now create a table and drop it in the impalad running in old mode. Following this create the same table from the new mode coodinator. It sometimes throws table not found exception.
> Test Create and Drop of a table in V1
> Running 1 iterations
> Going for : CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
> Going for : DROP TABLE PARTS;
> Test Create and Drop of a table in V2
> Running 1 iterations
> Going for : CREATE TABLE PARTS ( PART_ID DOUBLE, CREATE_TIME DOUBLE, LAST_ACCESS_TIME DOUBLE, PART_NAME STRING, SD_ID DOUBLE, TBL_ID DOUBLE) STORED AS PARQUETFILE;
> Traceback (most recent call last):
> File "testing.py", line 21, in execute_query
> cursor.execute(query)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
> configuration=configuration)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
> self._execute_async(op)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
> operation_fn()
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
> async=True)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
> return self._operation('ExecuteStatement', req)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
> resp = self._rpc(kind, request)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
> err_if_rpc_not_ok(response)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
> CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))
> Going for : DROP TABLE PARTS;
> Traceback (most recent call last):
> File "testing.py", line 21, in execute_query
> cursor.execute(query)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 302, in execute
> configuration=configuration)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
> self._execute_async(op)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
> operation_fn()
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 340, in op
> async=True)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 1027, in execute
> return self._operation('ExecuteStatement', req)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 957, in _operation
> resp = self._rpc(kind, request)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
> err_if_rpc_not_ok(response)
> File "/Users/parna/workspace/virtual_envs/metadata/lib/python2.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> HiveServer2Error: LocalCatalogException: Could not load table parnatest.parts from metastore
> CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[CatalogException: Table not found: parts]))
> Corresponding Impalad log will attached as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org