You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Venugopal Reddy K (Jira)" <ji...@apache.org> on 2024/05/31 11:13:00 UTC
[jira] [Updated] (IMPALA-13120) Failed table loads are not tried to load again even though hive metastore is UP
[ https://issues.apache.org/jira/browse/IMPALA-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Venugopal Reddy K updated IMPALA-13120:
---------------------------------------
Description:
*Description:*
If the metastore is down at the time when the table load is triggered, catalogd creates a new IncompleteTable instance with cause=TableLoadingException and updates catalog with a new version. And on coordinator/impalad, StmtMetadataLoader loadTables() that has been waiting for table load to complete, considers table as loaded/failed load. Then during the analyzer’s table resolve step, if the table is incomplete, TableLoadingException is thrown to user.
Note: IncompleteTable with cause not being null is considered as loaded.
*Henceforth, queries on the table doesn’t trigger the table load(at StmtMetadataLoader) since the table is IncompleteTable with non-null cause(i.e.,TableLoadingException). Even though metastore is UP later at some time, queries continue to fail with same TableLoadingException:*
{{CAUSED BY: TableLoadingException: Failed to load metadata for table: default.t1. Running 'invalidate metadata default.t1' may resolve this problem.}}
{{CAUSED BY: MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)}}
*At present, explicit Invalidate metadata is the only way to recover table from this state.* {*}Queries executed after metastore is up should succeed without the need for explicit invalidate metadata{*}{*}{{*}}
*Steps to Reproduce:*
# create a table from hive and insert some data into it.
# Bring down the hive metastore process
# Run a query on impala that triggers the table load. Query fails with TableLoadingException.
# Bring up the hive metastore process
# Run the query on impala again. It still fails with same TableLoadingException.
was:
*Description:*
If the metastore is down at the time when the table load is triggered, catalogd creates a new IncompleteTable instance with cause=TableLoadingException and updates catalog with a new version. And on coordinator/impalad, StmtMetadataLoader loadTables() that has been waiting for table load to complete, considers table as loaded/failed load. Then during the analyzer’s table resolve step, if the table is incomplete, TableLoadingException is thrown to user.
Note: IncompleteTable with cause not being null is considered as loaded.
*Henceforth, queries on the table doesn’t trigger the table load(at StmtMetadataLoader) since the table is IncompleteTable with non-null cause(i.e.,TableLoadingException). Even though metastore is UP later at some time, queries continue to fail with same TableLoadingException:*
{{CAUSED BY: TableLoadingException: Failed to load metadata for table: default.t1. Running 'invalidate metadata default.t1' may resolve this problem.}}
{{CAUSED BY: MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)}}
{{}}
*At present, explicit Invalidate metadata is the only way to recover table from this state.* {*}Queries executed after metastore is up should succeed without the need for explicit invalidate metadata{*}{*}{*}
*Steps to Reproduce:*
# create a table from hive and insert some data into it.
# Bring down the hive metastore process
# Run a query on impala that triggers the table load. Query fails with TableLoadingException.
# Bring up the hive metastore process
# Run the query on impala again. It still fails with same TableLoadingException.
> Failed table loads are not tried to load again even though hive metastore is UP
> -------------------------------------------------------------------------------
>
> Key: IMPALA-13120
> URL: https://issues.apache.org/jira/browse/IMPALA-13120
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Venugopal Reddy K
> Priority: Major
>
> *Description:*
> If the metastore is down at the time when the table load is triggered, catalogd creates a new IncompleteTable instance with cause=TableLoadingException and updates catalog with a new version. And on coordinator/impalad, StmtMetadataLoader loadTables() that has been waiting for table load to complete, considers table as loaded/failed load. Then during the analyzer’s table resolve step, if the table is incomplete, TableLoadingException is thrown to user.
> Note: IncompleteTable with cause not being null is considered as loaded.
> *Henceforth, queries on the table doesn’t trigger the table load(at StmtMetadataLoader) since the table is IncompleteTable with non-null cause(i.e.,TableLoadingException). Even though metastore is UP later at some time, queries continue to fail with same TableLoadingException:*
> {{CAUSED BY: TableLoadingException: Failed to load metadata for table: default.t1. Running 'invalidate metadata default.t1' may resolve this problem.}}
> {{CAUSED BY: MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)}}
> *At present, explicit Invalidate metadata is the only way to recover table from this state.* {*}Queries executed after metastore is up should succeed without the need for explicit invalidate metadata{*}{*}{{*}}
> *Steps to Reproduce:*
> # create a table from hive and insert some data into it.
> # Bring down the hive metastore process
> # Run a query on impala that triggers the table load. Query fails with TableLoadingException.
> # Bring up the hive metastore process
> # Run the query on impala again. It still fails with same TableLoadingException.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org