You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Andrew Sherman (Jira)" <ji...@apache.org> on 2022/08/19 23:00:00 UTC
[jira] (IMPALA-11502) Dropping files of Iceberg table in HadoopCatalog will cause DROP TABLE to fail
[ https://issues.apache.org/jira/browse/IMPALA-11502 ]
Andrew Sherman deleted comment on IMPALA-11502:
-----------------------------------------
was (Author: asherman):
Catalog log shows this exception
{code:java}
E0819 13:50:54.286012 1642964 ImpaladCatalog.java:255] Error adding catalog object: Table does not exist: drop_incomplete_table.iceberg_tbl
Java exception follows:
org.apache.impala.catalog.TableLoadingException: Table does not exist: drop_incomplete_table.iceberg_tbl
at org.apache.impala.catalog.iceberg.IcebergHadoopCatalog.loadTable(IcebergHadoopCatalog.java:88)
at org.apache.impala.util.IcebergUtil.loadTable(IcebergUtil.java:140)
at org.apache.impala.util.IcebergUtil.loadTable(IcebergUtil.java:129)
at org.apache.impala.catalog.IcebergTable.loadFromThrift(IcebergTable.java:437)
at org.apache.impala.catalog.Table.fromThrift(Table.java:538)
at org.apache.impala.catalog.ImpaladCatalog.addTable(ImpaladCatalog.java:469)
at org.apache.impala.catalog.ImpaladCatalog.addCatalogObject(ImpaladCatalog.java:324)
at org.apache.impala.catalog.ImpaladCatalog.updateCatalog(ImpaladCatalog.java:253)
at org.apache.impala.service.FeCatalogManager$CatalogdImpl.updateCatalogCache(FeCatalogManager.java:114)
at org.apache.impala.service.Frontend.updateCatalogCache(Frontend.java:521)
at org.apache.impala.service.JniFrontend.updateCatalogCache(JniFrontend.java:185)
W0819 13:53:45.886678 1640937 JvmPauseMonitor.java:205] Detected pause in JVM or host machine (eg GC): pause of approximately 10674ms
{code}
I think this failure causes the infinite loop [mentioned in the bug description|https://github.com/apache/impala/blob/2f74e956aa10db5af6a7cdc47e2ad42f63d5030f/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196]. The code is waiting to hear about the table, but the update never comes.
I tried to hack a fix for this by adding code in Table.fromThrift() to detect the missing missing iceberg metadata
{code:java}
if (thriftTable.isSetIceberg_table() && newTable instanceof FeIcebergTable) {
try {
IcebergUtil.loadTable((FeIcebergTable) newTable);
} catch (TableLoadingException tle) {
// Iceberg metadata is missing
LOG.warn("Could not load iceberg metadata:", tle);
newTable = IncompleteTable.createFailedMetadataLoadTable(
parentDb, thriftTable.getTbl_name(), tle);
}
}
{code}
but this causes the drop to fail
{code:java}
I0819 15:23:08.421619 1692452 jni-util.cc:286] e04f7a1c33fb6eb6:cae16f1c00000000] org.apache.impala.common.AnalysisException: org.apache.impala.catalog.TableLoadingException: Table does not exist: drop_incomplete_table.iceberg_tbl
at org.apache.impala.analysis.Analyzer.getTable(Analyzer.java:3499)
at org.apache.impala.analysis.InsertStmt.analyzeTargetTable(InsertStmt.java:457)
at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:325)
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:511)
at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468)
at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2005)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1913)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1737)
at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
{code}
> Dropping files of Iceberg table in HadoopCatalog will cause DROP TABLE to fail
> ------------------------------------------------------------------------------
>
> Key: IMPALA-11502
> URL: https://issues.apache.org/jira/browse/IMPALA-11502
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Gabor Kaszab
> Priority: Major
> Labels: iceberg, impala-iceberg
>
> When the files of n Iceberg table are dropped then a DROP TABLE will result in an error while the table will still show up in SHOW TABLES
> Here are the steps to repro:
> 1) Run from Impala-shell
> {code:java}
> DROP DATABASE IF EXISTS `drop_incomplete_table` CASCADE;
> CREATE DATABASE `drop_incomplete_table`;
> CREATE TABLE drop_incomplete_table.iceberg_tbl (i int) stored as iceberg
> tblproperties('iceberg.catalog'='hadoop.catalog',
> 'iceberg.catalog_location'='/test-warehouse/drop_incomplete_table');
> INSERT INTO drop_incomplete_table.iceberg_tbl VALUES (1), (2), (3);
> {code}
> 2) Drop the folder of the table with hdfs dfs
> {code:java}
> hdfs dfs -rm -r hdfs://localhost:20500/test-warehouse/drop_incomplete_table
> {code}
> 3) Try to drop the table from Impala-shell
> {code:java}
> DROP TABLE drop_incomplete_table.iceberg_tbl;
> {code}
> This results in the following error:
> {code:java}
> ERROR: TableNotFoundException: Table 'drop_incomplete_table.iceberg_tbl' does not exist in Iceberg catalog.
> {code}
> While table is still there in show tables output even after an invalidate metadata.
> Note, it's important for the repro to execute some SQL against the newly created table to load it in Impala. In this case I used an INSERT INTO but e.g. an ALTER TABLE would also be good. Apparently, when the table is "incomplete" (this is the state right after running CREATE TABLE) this works fine but not if the table is loaded.
> The suspicious part of code is in StmtMetadataLoader.loadTables() and getMissingTables() where there is a distinction between loaded and Incomplete tables.
> https://github.com/apache/impala/blob/2f74e956aa10db5af6a7cdc47e2ad42f63d5030f/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org